-- Logs begin at Thu 2022-05-19 11:17:16 UTC, end at Thu 2022-05-19 12:43:09 UTC. -- May 19 11:56:25.021782 dv-cinder-804757-28-iscsi systemd[1]: Starting Devstack devstack@keystone.service... May 19 11:56:25.032418 dv-cinder-804757-28-iscsi devstack@keystone.service[80530]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini May 19 11:56:25.034572 dv-cinder-804757-28-iscsi devstack@keystone.service[80530]: *** Starting uWSGI 2.0.18-debian (64bit) on [Thu May 19 11:56:25 2022] *** May 19 11:56:25.034572 dv-cinder-804757-28-iscsi devstack@keystone.service[80530]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 May 19 11:56:25.034572 dv-cinder-804757-28-iscsi devstack@keystone.service[80530]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 May 19 11:56:25.034572 dv-cinder-804757-28-iscsi devstack@keystone.service[80530]: nodename: dv-cinder-804757-28-iscsi May 19 11:56:25.034572 dv-cinder-804757-28-iscsi devstack@keystone.service[80530]: machine: x86_64 May 19 11:56:25.034572 dv-cinder-804757-28-iscsi devstack@keystone.service[80530]: clock source: unix May 19 11:56:25.034572 dv-cinder-804757-28-iscsi devstack@keystone.service[80530]: pcre jit disabled May 19 11:56:25.034572 dv-cinder-804757-28-iscsi devstack@keystone.service[80530]: detected number of CPU cores: 4 May 19 11:56:25.034572 dv-cinder-804757-28-iscsi devstack@keystone.service[80530]: current working directory: / May 19 11:56:25.034572 dv-cinder-804757-28-iscsi devstack@keystone.service[80530]: detected binary path: /usr/bin/uwsgi-core May 19 11:56:25.034572 dv-cinder-804757-28-iscsi devstack@keystone.service[80530]: your processes number limit is 47680 May 19 11:56:25.034572 dv-cinder-804757-28-iscsi devstack@keystone.service[80530]: your memory page size is 4096 bytes May 19 11:56:25.034572 dv-cinder-804757-28-iscsi devstack@keystone.service[80530]: detected max file descriptor number: 2048 May 19 11:56:25.034572 dv-cinder-804757-28-iscsi devstack@keystone.service[80530]: lock engine: pthread robust mutexes May 19 11:56:25.035325 dv-cinder-804757-28-iscsi devstack@keystone.service[80530]: thunder lock: enabled May 19 11:56:25.035325 dv-cinder-804757-28-iscsi devstack@keystone.service[80530]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 May 19 11:56:25.035325 dv-cinder-804757-28-iscsi devstack@keystone.service[80530]: Python version: 3.8.10 (default, Mar 15 2022, 12:22:08) [GCC 9.4.0] May 19 11:56:25.081814 dv-cinder-804757-28-iscsi devstack@keystone.service[80530]: Python main interpreter initialized at 0x561498dfc9c0 May 19 11:56:25.081814 dv-cinder-804757-28-iscsi devstack@keystone.service[80530]: python threads support enabled May 19 11:56:25.081814 dv-cinder-804757-28-iscsi devstack@keystone.service[80530]: your server socket listen backlog is limited to 100 connections May 19 11:56:25.081814 dv-cinder-804757-28-iscsi devstack@keystone.service[80530]: your mercy for graceful operations on workers is 90 seconds May 19 11:56:25.082178 dv-cinder-804757-28-iscsi devstack@keystone.service[80530]: mapped 403077 bytes (393 KB) for 2 cores May 19 11:56:25.082252 dv-cinder-804757-28-iscsi devstack@keystone.service[80530]: *** Operational MODE: preforking *** May 19 11:56:25.082589 dv-cinder-804757-28-iscsi devstack@keystone.service[80530]: *** uWSGI is running in multiple interpreter mode *** May 19 11:56:25.082589 dv-cinder-804757-28-iscsi devstack@keystone.service[80530]: spawned uWSGI master process (pid: 80530) May 19 11:56:25.082801 dv-cinder-804757-28-iscsi systemd[1]: Started Devstack devstack@keystone.service. May 19 11:56:25.083894 dv-cinder-804757-28-iscsi devstack@keystone.service[80530]: spawned uWSGI worker 1 (pid: 80539, cores: 1) May 19 11:56:25.083894 dv-cinder-804757-28-iscsi devstack@keystone.service[80530]: spawned uWSGI worker 2 (pid: 80540, cores: 1) May 19 11:56:25.083894 dv-cinder-804757-28-iscsi devstack@keystone.service[80530]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... May 19 11:56:31.000346 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 19 11:56:31.000346 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: from cryptography.utils import int_from_bytes May 19 11:56:31.001097 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 19 11:56:31.001097 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: from cryptography.utils import int_from_bytes May 19 11:56:31.001238 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 19 11:56:31.001238 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: from cryptography.utils import int_from_bytes May 19 11:56:31.001660 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 19 11:56:31.001660 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: from cryptography.utils import int_from_bytes May 19 11:56:31.051521 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} May 19 11:56:31.051705 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} May 19 11:56:31.051813 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} May 19 11:56:31.051915 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} May 19 11:56:31.051995 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] command line args: [] {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} May 19 11:56:31.052071 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] command line args: [] {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} May 19 11:56:31.052180 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} May 19 11:56:31.052316 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} May 19 11:56:31.052413 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] ================================================================================ {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} May 19 11:56:31.052496 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] ================================================================================ {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} May 19 11:56:31.052623 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] admin_token = **** {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:31.052709 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] admin_token = **** {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:31.052869 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] config_dir = [] {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:31.052956 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] config_dir = [] {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:31.053094 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:31.053197 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:31.053278 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] config_source = [] {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:31.053357 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] config_source = [] {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:31.053434 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] debug = True {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:31.053515 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] debug = True {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:31.053592 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:31.053885 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:31.053976 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:31.054041 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:31.054109 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:31.054168 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:31.054356 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:31.054443 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:31.054586 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:31.054671 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:31.054809 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] list_limit = None {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:31.054892 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] list_limit = None {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:31.054967 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] log_config_append = None {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:31.055033 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] log_config_append = None {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:31.055185 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:31.055275 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:31.055336 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] log_dir = None {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:31.055397 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] log_dir = None {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:31.055459 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] log_file = None {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:31.055521 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] log_file = None {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:31.055732 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:31.055856 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:31.055983 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:31.056099 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:31.056213 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:31.056313 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:31.056460 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:31.056575 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:31.056656 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:31.056726 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:31.056790 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:31.056855 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:31.057118 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:31.057218 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:31.057291 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(user_domain)s %(project_domain)s {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:31.057360 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(user_domain)s %(project_domain)s {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:31.057493 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:31.057606 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:31.057724 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:31.057819 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:31.057951 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:31.058037 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:31.058176 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:31.058266 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:31.058415 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:31.058498 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:31.058656 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:31.058738 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:31.058930 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:31.059013 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:31.059415 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.23/identity {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:31.059496 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.23/identity {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:31.059605 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] publish_errors = False {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:31.059711 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] publish_errors = False {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:31.059841 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:31.059931 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:31.060065 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:31.060167 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:31.060252 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:31.060329 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:31.060469 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:31.060657 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:31.060740 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:31.060814 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:31.060962 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] standard_threads = False {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:31.061138 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] standard_threads = False {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:31.061138 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:31.061284 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:31.061365 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:31.061437 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:31.061512 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:31.062073 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:31.062269 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] use_journal = False {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:31.062377 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] use_journal = False {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:31.062377 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] use_json = False {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:31.062547 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] use_json = False {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:31.062547 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] use_stderr = False {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:31.062708 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] use_stderr = False {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:31.062781 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] use_syslog = False {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:31.062939 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] use_syslog = False {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:31.063125 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:31.063206 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 11:56:31.063381 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.063475 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.063620 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.063697 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.063906 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.063999 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.064147 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.064230 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.064392 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.064469 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.064626 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.064705 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.064871 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.064963 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.065102 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.065179 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.065344 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.065428 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.065588 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.065670 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.065834 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.065915 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.066068 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.066152 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.066294 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.066435 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.066506 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.066571 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.066757 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.066851 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.067003 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.067087 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.067238 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.067316 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.067491 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.067577 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.067744 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.067850 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.068012 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.068089 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.068242 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.068326 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.068493 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.068575 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.068749 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.069084 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.069251 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.069345 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.069500 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.069587 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.069730 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.069816 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.069960 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.070043 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.070189 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.070273 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.070423 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.070509 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.070669 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.070782 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.070913 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.071017 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.071140 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.071255 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.071359 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.071461 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.071596 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.071719 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.071883 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.071974 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.072083 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.072213 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.072354 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.072495 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.072619 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.072752 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.072874 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.072997 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.073118 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.073254 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.073376 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.073510 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.073643 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.073774 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.073909 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.074033 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.074162 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.074294 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.074418 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.074549 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.074671 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.074795 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.074915 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.075040 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.075174 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.075297 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.075418 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.075554 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.075678 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.075842 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.075983 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.076107 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.076107 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.076307 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.076307 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.076487 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.076613 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.076697 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.076776 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.076853 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.076925 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.077048 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.077135 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.077915 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.078046 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.078147 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.078251 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.078386 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.078459 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.078525 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.078731 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.078731 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.078867 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.078938 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.079033 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.079186 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.079310 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.079394 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.079473 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.079601 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.079682 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.079862 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.079947 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.080101 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.080180 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.080372 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.080464 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.080611 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.080692 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.080860 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.080945 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.081106 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.081196 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.081339 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.081424 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.081572 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.081651 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.081804 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.081894 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.082035 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.082122 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.082273 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.082351 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.082505 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.082583 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.082740 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.082862 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.083041 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.083123 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.083195 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.083264 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.083421 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.083511 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.083653 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.083735 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.083911 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.083993 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.084143 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.084230 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.084443 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.084536 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.084680 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.084761 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.084918 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.085006 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.085158 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.085244 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.085395 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.085476 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.085628 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.085716 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.085865 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.085943 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.086094 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.086179 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.086318 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.086406 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.086544 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.086623 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.086768 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.086847 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.086993 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.087080 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.087217 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.087296 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.087453 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.087531 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.087684 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.087771 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.087925 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.088000 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.088185 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.088264 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.088562 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.088648 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.088814 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.088892 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.089048 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] auth.external = None {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.089140 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.089276 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] auth.external = None {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.089357 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.089566 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.089645 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.089800 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.089889 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.090032 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] auth.password = None {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.090179 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.090264 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] auth.token = None {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.090339 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] auth.password = None {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.090489 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.090569 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] auth.token = None {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.090726 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.090813 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.090974 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.091056 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.091207 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.091286 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.091434 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.091513 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.091682 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.091764 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.091935 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.092018 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.092172 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] credential.caching = True {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.092252 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.092403 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] credential.caching = True {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.092480 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.092633 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.092714 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.092912 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.093077 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.093331 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.093403 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.093571 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.093655 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.093815 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.093896 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.094076 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.094190 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.094345 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.094447 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.094600 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.094680 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.094836 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.094918 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.095087 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.095199 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.095371 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.095457 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.095621 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.095859 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.095953 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.096026 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.096185 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.096303 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.096441 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.096536 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] federation.caching = True {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.096675 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] federation.caching = True {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.096756 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.096910 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.096991 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.097153 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.097236 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.097382 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.097463 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.097610 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.097689 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.097853 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.097937 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.098083 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.098163 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.098328 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.098414 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.098561 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.098647 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.098801 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.098880 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.099032 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.099122 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.099269 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] identity.caching = True {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.099349 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.099497 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] identity.caching = True {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.099575 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.099741 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.099853 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.100001 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.100081 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.100236 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.100316 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.100470 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.100558 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.100715 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.100804 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.101040 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.101120 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.101275 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.101362 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.101522 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.101603 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.101752 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.101831 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.101983 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.102071 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.102206 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.102285 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.102433 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.102513 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.102674 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.102762 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.102919 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.102998 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.103159 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.103238 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.103404 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.103492 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.103644 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.103724 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.103921 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.104004 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.104158 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.104249 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.104387 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.104473 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.104614 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.104694 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.104839 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.104927 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.105063 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.105142 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.105295 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.105376 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.105531 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.105622 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.105766 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.105847 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.105999 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.106078 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.106231 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.106318 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.106477 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.106557 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.106717 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.106796 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.106949 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.107036 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.107192 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.107290 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.107439 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.107520 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.107667 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.107755 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.107915 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.107996 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.108154 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.108233 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.108386 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.108606 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.108694 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.108756 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.108876 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.108946 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.109173 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.109315 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.109446 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.109575 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.109704 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.109833 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.109961 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.110106 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.110236 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.110365 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.110499 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.110639 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.110770 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.110908 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.111043 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.111173 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.111307 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.111436 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.111572 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.111656 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.111856 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.111982 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] ldap.user = None {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.112107 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] ldap.user = None {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.112231 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.112388 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.112468 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.112540 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.112612 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.112683 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.112754 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.112833 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.112907 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.112979 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.113050 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.113121 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.113192 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.113270 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.113350 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.113489 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.113570 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.113721 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.113801 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.113954 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.114036 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.114187 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.114267 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.114454 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.114560 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.114698 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.114782 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.114941 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.115021 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.115187 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.115267 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.115432 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.115514 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.115658 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.115737 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.115927 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.116015 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.116163 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.116244 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.116394 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.116472 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.116625 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.116703 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.116853 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.116933 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.117083 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.117163 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.117325 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.117405 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.117561 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.117643 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.117791 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.117870 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.118027 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.118106 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.118254 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.118335 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.118525 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.118612 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.118758 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.118839 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.118991 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.119073 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.119224 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.119304 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.119467 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.119546 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.119696 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.119776 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.119951 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.120039 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.120186 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.120267 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] resource.caching = True {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.120425 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] resource.caching = True {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.120506 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.120668 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.120755 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.120903 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.120983 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.121134 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.121214 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.121379 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.121459 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.121618 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.121698 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.121847 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.121927 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.122087 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.122166 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.122318 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.122396 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.122549 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] role.caching = True {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.122630 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.122777 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] role.caching = True {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.122856 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] role.driver = sql {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.123010 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.123089 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] role.driver = sql {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.123234 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.123318 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.123478 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.123558 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.123721 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.123860 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.123992 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.124634 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.124759 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.125004 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.125182 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.125255 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.125385 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.125510 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.125646 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.125776 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.125905 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.126042 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.126171 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.126301 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.126429 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.126558 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.126687 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.126824 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.126953 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.127082 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.127209 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.127348 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.127478 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.127615 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.127744 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.127889 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.128014 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.128140 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.128263 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.128401 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.128527 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.128653 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.128776 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.128900 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.129038 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.129172 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.129297 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.129423 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.129547 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.129672 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.129804 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.129930 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.130054 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.130208 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.130335 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.130465 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.130602 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.130741 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.130867 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.130993 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.131117 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.131242 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.131374 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.131501 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.131626 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.131751 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.131892 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.132018 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.132152 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.132278 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] token.caching = True {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.132402 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] token.caching = True {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.132538 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.132663 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.132789 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.132922 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.133047 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.133172 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.133296 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.133420 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.133545 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.133678 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.133804 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.133929 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.134056 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.134190 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.134316 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.134448 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.134575 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.134699 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.134823 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.134948 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.135072 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.135163 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.135227 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.135359 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.135520 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.135645 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.135780 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.135946 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.136078 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.136205 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.136334 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.136461 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.136588 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.136724 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.136854 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] database.connection = **** {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.136991 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] database.connection = **** {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.137203 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.137341 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.137468 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.137592 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.137720 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.137865 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.138004 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.138130 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.138262 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.138391 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.138520 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.138647 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.138783 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.138912 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.139041 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.139170 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.139299 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.139427 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.139564 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.140344 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.140535 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.140625 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.140690 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.140749 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.140816 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.140878 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.141080 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.141207 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.141331 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.141455 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.141587 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.141712 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.141847 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.141972 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.142096 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.142096 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80540) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} May 19 11:56:31.142343 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 11:56:31.142343 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80539) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} May 19 11:56:31.173013 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80540) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 11:56:31.173298 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80540) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 11:56:31.173438 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80540) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 11:56:31.173594 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80539) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 11:56:31.173735 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80539) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 11:56:31.173910 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80539) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 11:56:31.182362 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80540) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 11:56:31.182828 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80539) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 11:56:31.211882 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80540) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 11:56:31.212091 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80540) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 11:56:31.212343 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80539) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 11:56:31.212579 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80539) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 11:56:31.266567 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80539) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 11:56:31.266852 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80540) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 11:56:31.319498 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80539) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 11:56:31.319745 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80539) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 11:56:31.320201 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80540) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 11:56:31.320436 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80540) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 11:56:31.387050 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80539) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 11:56:31.387340 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80539) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 11:56:31.387477 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80539) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 11:56:31.387731 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80539) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 11:56:31.387977 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80539) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 11:56:31.388196 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80539) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 11:56:31.389120 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80540) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 11:56:31.389353 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80540) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 11:56:31.389576 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80540) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 11:56:31.389820 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80540) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 11:56:31.390039 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80540) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 11:56:31.390422 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80540) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 11:56:31.411837 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80539) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 11:56:31.412010 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80540) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 11:56:31.412138 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80539) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 11:56:31.412261 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80540) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 11:56:31.412386 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80539) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 11:56:31.412522 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80540) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 11:56:31.412647 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80539) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 11:56:31.412782 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80540) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 11:56:31.412905 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80539) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 11:56:31.413027 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80540) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 11:56:31.413158 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80539) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 11:56:31.413280 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80540) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 11:56:31.413403 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80539) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 11:56:31.413524 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80540) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 11:56:31.425321 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80540) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 11:56:31.425549 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80539) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 11:56:31.440729 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80539) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 11:56:31.440948 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80540) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 11:56:31.441102 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80539) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 11:56:31.441236 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80540) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 11:56:31.441382 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80539) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 11:56:31.441512 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80540) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 11:56:31.441639 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80539) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 11:56:31.441768 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80540) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 11:56:31.441906 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80539) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 11:56:31.442032 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80540) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 11:56:31.442167 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80539) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 11:56:31.442293 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80540) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 11:56:31.442434 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80539) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 11:56:31.442561 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80540) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 11:56:31.482178 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80539) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 11:56:31.482419 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80540) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 11:56:31.482557 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80539) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 19 11:56:31.482688 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80540) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 19 11:56:31.482814 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80539) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 11:56:31.482939 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80539) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 19 11:56:31.483065 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80540) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 11:56:31.483065 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80540) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 19 11:56:31.483307 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80539) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 11:56:31.483429 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80540) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 11:56:31.484290 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80539) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 11:56:31.484393 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80540) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 11:56:31.484528 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80539) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 11:56:31.484609 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80540) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 11:56:31.484777 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80539) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 11:56:31.484899 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80540) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 11:56:31.495695 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80539) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 11:56:31.495915 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80540) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 11:56:31.496143 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80539) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 11:56:31.496280 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80540) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 11:56:31.496405 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80539) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 11:56:31.496540 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80540) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 11:56:31.496665 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80539) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 11:56:31.496798 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80540) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 11:56:31.505449 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80539) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 11:56:31.505772 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80540) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 11:56:31.508862 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80539) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 11:56:31.509139 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80539) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 11:56:31.509281 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80540) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 11:56:31.509407 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80539) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 11:56:31.509530 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80540) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 11:56:31.509802 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80539) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 11:56:31.509944 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80540) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 11:56:31.510092 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80539) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 11:56:31.510228 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80540) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 11:56:31.510352 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80539) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 11:56:31.510475 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80540) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 11:56:31.510597 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80539) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 11:56:31.510719 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80540) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 11:56:31.510848 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80539) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 11:56:31.510973 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80540) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 11:56:31.511121 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80539) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 11:56:31.511256 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80540) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 11:56:31.511381 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80540) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 11:56:31.511511 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80539) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 11:56:31.511634 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80539) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 11:56:31.511755 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80540) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 11:56:31.511912 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80540) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 11:56:31.539013 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80539) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 11:56:31.539379 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80540) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 11:56:31.540656 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80539) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 11:56:31.540937 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80539) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 11:56:31.541022 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80540) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 11:56:31.541277 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80540) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 11:56:31.547215 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80539) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 11:56:31.547337 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80540) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 11:56:31.547495 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80539) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 11:56:31.547579 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80540) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 11:56:31.547775 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80539) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 11:56:31.547875 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80540) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 11:56:31.555099 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80539) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 11:56:31.555249 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80540) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 11:56:31.555380 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80539) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 11:56:31.555530 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80540) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 11:56:31.560113 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80539) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 11:56:31.560206 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80540) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 11:56:31.560401 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80539) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 11:56:31.560486 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80540) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 11:56:31.560684 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80539) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 11:56:31.560771 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80540) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 11:56:31.560964 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80539) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 11:56:31.561055 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80540) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 11:56:31.561234 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80540) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 11:56:31.561315 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80539) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 11:56:31.561503 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80539) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 11:56:31.561584 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80540) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 11:56:31.561808 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80540) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 11:56:31.562325 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80539) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 11:56:31.562414 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80540) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 11:56:31.562489 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80539) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 11:56:31.581296 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80540) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 11:56:31.581569 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80540) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 11:56:31.581708 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80539) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 11:56:31.581876 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80539) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 11:56:31.586214 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80540) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 11:56:31.586437 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80539) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 11:56:31.589985 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80540) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 11:56:31.590188 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80539) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 11:56:31.590318 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80540) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 11:56:31.590480 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80539) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 11:56:31.595952 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80540) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 11:56:31.596171 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80539) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 11:56:31.599662 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80540) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 11:56:31.599813 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80539) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 11:56:31.603476 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80540) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 11:56:31.603611 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80539) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 11:56:31.603772 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80540) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 11:56:31.603930 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80539) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 11:56:31.604080 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80540) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 11:56:31.604214 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80539) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 11:56:31.604362 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80540) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 11:56:31.604499 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80539) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 11:56:31.604624 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80540) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 11:56:31.604748 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80539) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 11:56:31.604924 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80540) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 11:56:31.605060 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80539) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 11:56:31.624936 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80540) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 11:56:31.625305 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80540) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 11:56:31.625575 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80540) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 11:56:31.625706 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80539) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 11:56:31.625829 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80540) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 11:56:31.625964 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80539) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 11:56:31.626226 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80539) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 11:56:31.626501 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80539) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 11:56:31.633966 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80540) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 11:56:31.634499 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80539) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 11:56:31.635678 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80540) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 11:56:31.635983 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80540) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 11:56:31.636260 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80539) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 11:56:31.636540 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80539) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 11:56:31.639031 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80540) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 11:56:31.639345 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80540) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 11:56:31.639576 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80539) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 11:56:31.639721 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80540) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 11:56:31.639974 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80540) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 11:56:31.640130 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80539) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 11:56:31.640300 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80540) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 11:56:31.640432 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80539) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 11:56:31.640633 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80539) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 11:56:31.640904 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80539) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 11:56:31.653939 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80540) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 11:56:31.654224 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80540) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 11:56:31.654502 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80540) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 11:56:31.654646 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80539) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 11:56:31.654775 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80540) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 11:56:31.654902 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80539) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 11:56:31.655031 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80540) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 11:56:31.655169 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80539) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 11:56:31.655305 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80540) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 11:56:31.656016 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80539) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 11:56:31.656195 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80539) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 11:56:31.656460 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80540) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 11:56:31.656600 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80539) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 11:56:31.656744 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80539) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 11:56:31.675390 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80540) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 11:56:31.675536 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80539) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 11:56:31.678692 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80540) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 11:56:31.678848 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80539) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 11:56:31.682000 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80540) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 11:56:31.682182 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80539) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 11:56:31.683607 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80540) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 11:56:31.683776 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80539) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 11:56:31.685260 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80540) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 11:56:31.685537 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80540) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 11:56:31.685676 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80539) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 11:56:31.685809 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80540) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 11:56:31.685941 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80539) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 11:56:31.686098 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80539) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 11:56:31.693447 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80540) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 11:56:31.693656 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80539) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 11:56:31.695323 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80540) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 11:56:31.695460 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80539) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 11:56:31.698739 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80540) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 11:56:31.698903 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80539) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 11:56:31.699043 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80540) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 11:56:31.699171 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80539) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 11:56:31.699299 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80540) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 11:56:31.699426 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80539) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 11:56:31.699567 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80539) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 11:56:31.699727 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80540) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 11:56:31.708860 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80539) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 11:56:31.709195 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80539) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 11:56:31.709325 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80540) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 11:56:31.709519 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80539) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 11:56:31.709662 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80540) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 11:56:31.709816 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80540) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 11:56:31.717798 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80540) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 11:56:31.718677 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80540) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 11:56:31.719052 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80540) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 11:56:31.719247 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80540) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 11:56:31.719515 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80540) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 11:56:31.719803 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80540) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 11:56:31.719935 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80539) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 11:56:31.720093 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80540) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 11:56:31.720224 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80539) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 11:56:31.720362 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80540) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 11:56:31.720490 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80539) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 11:56:31.720625 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80540) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 11:56:31.720752 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80539) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 11:56:31.720879 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80540) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 11:56:31.721015 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80539) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 11:56:31.721154 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80540) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 11:56:31.721283 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80539) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 11:56:31.721410 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80540) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 11:56:31.721537 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80539) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 11:56:31.721663 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80540) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 11:56:31.721799 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80539) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 11:56:31.721927 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80540) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 11:56:31.722053 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80539) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 11:56:31.722242 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80539) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 11:56:31.722522 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80539) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 11:56:31.722786 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80539) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 11:56:31.723057 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80539) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 11:56:31.723316 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80539) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 11:56:31.761662 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x561498dfc9c0 pid: 80540 (default app) May 19 11:56:31.763420 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x561498dfc9c0 pid: 80539 (default app) May 19 11:56:31.765203 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ef53dc-3746-4de1-a2db-a7f924da8274 None None] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:56:31.765505 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ef53dc-3746-4de1-a2db-a7f924da8274 None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:56:31.765786 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ef53dc-3746-4de1-a2db-a7f924da8274 None None] PATH_INFO: `/v3` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:56:31.769954 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1/1] 10.222.0.23 () {50 vars in 791 bytes} [Thu May 19 11:56:31 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:56:41.005297 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3faa82c-8673-409d-9738-f0a58cdae511 None None] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:56:41.005807 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3faa82c-8673-409d-9738-f0a58cdae511 None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:56:41.005887 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3faa82c-8673-409d-9738-f0a58cdae511 None None] PATH_INFO: `/` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:56:41.010147 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1/2] 10.222.0.23 () {54 vars in 926 bytes} [Thu May 19 11:56:41 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 11:56:41.015637 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c769c36-6d31-4af9-89d3-668e330cbae2 None None] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:56:41.016099 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c769c36-6d31-4af9-89d3-668e330cbae2 None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:56:41.016227 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c769c36-6d31-4af9-89d3-668e330cbae2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:56:41.087250 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: INFO dbcounter [None req-4c769c36-6d31-4af9-89d3-668e330cbae2 None None] Registered counter for database keystone May 19 11:56:41.109543 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG oslo_db.sqlalchemy.engines [None req-4c769c36-6d31-4af9-89d3-668e330cbae2 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80540) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} May 19 11:56:41.115912 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writer thread running {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} May 19 11:56:41.290283 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG passlib.handlers.bcrypt [None req-4c769c36-6d31-4af9-89d3-668e330cbae2 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80540) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} May 19 11:56:41.290628 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG passlib.handlers.bcrypt [None req-4c769c36-6d31-4af9-89d3-668e330cbae2 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80540) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} May 19 11:56:41.346554 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.auth.core [None req-4c769c36-6d31-4af9-89d3-668e330cbae2 None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 11:56:41.348193 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-4c769c36-6d31-4af9-89d3-668e330cbae2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 11:56:41.472363 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 2/3] 10.222.0.23 () {58 vars in 1023 bytes} [Thu May 19 11:56:41 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 458 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) May 19 11:56:41.655553 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3faa82c-8673-409d-9738-f0a58cdae511 None None] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:56:41.726424 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: INFO dbcounter [None req-b3faa82c-8673-409d-9738-f0a58cdae511 None None] Registered counter for database keystone May 19 11:56:41.746363 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG oslo_db.sqlalchemy.engines [None req-b3faa82c-8673-409d-9738-f0a58cdae511 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80539) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} May 19 11:56:41.753352 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writer thread running {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} May 19 11:56:41.872877 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3faa82c-8673-409d-9738-f0a58cdae511 None None] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:56:41.877882 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2406fc2-e8fa-4581-b3a0-1a7c96302a70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:56:41.878549 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2406fc2-e8fa-4581-b3a0-1a7c96302a70 admin admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:56:41.878833 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2406fc2-e8fa-4581-b3a0-1a7c96302a70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:56:41.879111 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2406fc2-e8fa-4581-b3a0-1a7c96302a70 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:56:41.887228 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2406fc2-e8fa-4581-b3a0-1a7c96302a70 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:56:41.939012 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: WARNING keystone.common.rbac_enforcer.enforcer [None req-f2406fc2-e8fa-4581-b3a0-1a7c96302a70 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. May 19 11:56:41.939244 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2406fc2-e8fa-4581-b3a0-1a7c96302a70 admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:56:41.949747 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 2/4] 10.222.0.23 () {58 vars in 1324 bytes} [Thu May 19 11:56:41 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) May 19 11:56:41.955852 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c769c36-6d31-4af9-89d3-668e330cbae2 None None] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:56:41.970109 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c769c36-6d31-4af9-89d3-668e330cbae2 None None] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:56:41.974594 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83c88b0-3c81-4bbe-912e-458b204eedc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:56:41.975693 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83c88b0-3c81-4bbe-912e-458b204eedc0 admin admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:56:41.976093 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83c88b0-3c81-4bbe-912e-458b204eedc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:56:41.976386 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83c88b0-3c81-4bbe-912e-458b204eedc0 admin admin] PATH_INFO: `/v3/projects/f3569780f2fe4b24b4109e2eee3c26fe` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:56:41.977904 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83c88b0-3c81-4bbe-912e-458b204eedc0 admin admin] RBAC: Authorizing `identity:get_project(project_id=f3569780f2fe4b24b4109e2eee3c26fe)` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:56:42.140502 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: WARNING keystone.common.rbac_enforcer.enforcer [None req-f83c88b0-3c81-4bbe-912e-458b204eedc0 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. May 19 11:56:42.141129 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83c88b0-3c81-4bbe-912e-458b204eedc0 admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:56:42.145841 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 3/5] 10.222.0.23 () {56 vars in 1207 bytes} [Thu May 19 11:56:41 2022] GET /identity/v3/projects/f3569780f2fe4b24b4109e2eee3c26fe => generated 344 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:56:44.378659 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdfea4e-14dd-4e5f-95aa-53e81c4e15b0 None None] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:56:44.379197 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdfea4e-14dd-4e5f-95aa-53e81c4e15b0 None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:56:44.379197 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdfea4e-14dd-4e5f-95aa-53e81c4e15b0 None None] PATH_INFO: `/` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:56:44.383541 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 3/6] 10.222.0.23 () {54 vars in 926 bytes} [Thu May 19 11:56:44 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 11:56:44.389098 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829c21f7-c42d-4baf-90dd-ef89f760bfde None None] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:56:44.389521 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829c21f7-c42d-4baf-90dd-ef89f760bfde None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:56:44.389649 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829c21f7-c42d-4baf-90dd-ef89f760bfde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:56:44.409080 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.auth.core [None req-829c21f7-c42d-4baf-90dd-ef89f760bfde None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 11:56:44.410602 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-829c21f7-c42d-4baf-90dd-ef89f760bfde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 11:56:44.418441 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 4/7] 10.222.0.23 () {58 vars in 1023 bytes} [Thu May 19 11:56:44 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) May 19 11:56:44.440942 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cdfea4e-14dd-4e5f-95aa-53e81c4e15b0 None None] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:56:44.453505 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cdfea4e-14dd-4e5f-95aa-53e81c4e15b0 None None] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:56:44.458513 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15807b88-b0f6-4f02-9895-e17a8d2b2844 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:56:44.459149 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15807b88-b0f6-4f02-9895-e17a8d2b2844 admin admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:56:44.459435 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15807b88-b0f6-4f02-9895-e17a8d2b2844 admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:56:44.459713 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15807b88-b0f6-4f02-9895-e17a8d2b2844 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:56:44.467411 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15807b88-b0f6-4f02-9895-e17a8d2b2844 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:56:44.469229 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15807b88-b0f6-4f02-9895-e17a8d2b2844 admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:56:44.478122 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 4/8] 10.222.0.23 () {58 vars in 1324 bytes} [Thu May 19 11:56:44 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 19 11:56:44.483527 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829c21f7-c42d-4baf-90dd-ef89f760bfde None None] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:56:44.493464 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829c21f7-c42d-4baf-90dd-ef89f760bfde None None] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:56:44.497836 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8447e1b4-cd22-4bdd-9a94-882c690a2a51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:56:44.498697 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8447e1b4-cd22-4bdd-9a94-882c690a2a51 admin admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:56:44.498981 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8447e1b4-cd22-4bdd-9a94-882c690a2a51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:56:44.499257 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8447e1b4-cd22-4bdd-9a94-882c690a2a51 admin admin] PATH_INFO: `/v3/users/11855e565a88475189d279de9469c271` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:56:44.500669 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8447e1b4-cd22-4bdd-9a94-882c690a2a51 admin admin] RBAC: Authorizing `identity:get_user(user_id=11855e565a88475189d279de9469c271)` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:56:44.503149 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8447e1b4-cd22-4bdd-9a94-882c690a2a51 admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:56:44.504744 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 5/9] 10.222.0.23 () {56 vars in 1198 bytes} [Thu May 19 11:56:44 2022] GET /identity/v3/users/11855e565a88475189d279de9469c271 => generated 247 bytes in 22 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:56:46.804755 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829ec70e-e5e5-44a9-ae09-dc95aecb6cde None None] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:56:46.805277 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829ec70e-e5e5-44a9-ae09-dc95aecb6cde None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:56:46.805361 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829ec70e-e5e5-44a9-ae09-dc95aecb6cde None None] PATH_INFO: `/` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:56:46.806973 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 5/10] 10.222.0.23 () {54 vars in 926 bytes} [Thu May 19 11:56:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 11:56:46.807763 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e07d0f-6f49-4809-95bf-338a6158f862 None None] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:56:46.808164 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e07d0f-6f49-4809-95bf-338a6158f862 None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:56:46.808426 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e07d0f-6f49-4809-95bf-338a6158f862 None None] PATH_INFO: `/` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:56:46.809787 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 6/11] 10.222.0.23 () {54 vars in 926 bytes} [Thu May 19 11:56:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 11:56:46.811913 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3364db8e-f975-4b36-a6b7-62f3423f63a4 None None] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:56:46.812209 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3364db8e-f975-4b36-a6b7-62f3423f63a4 None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:56:46.812496 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3364db8e-f975-4b36-a6b7-62f3423f63a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:56:46.814897 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fc5d44-5593-4098-93d9-9ea257e45de6 None None] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:56:46.815191 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fc5d44-5593-4098-93d9-9ea257e45de6 None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:56:46.815473 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fc5d44-5593-4098-93d9-9ea257e45de6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:56:46.831840 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.auth.core [None req-45fc5d44-5593-4098-93d9-9ea257e45de6 None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 11:56:46.833013 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-45fc5d44-5593-4098-93d9-9ea257e45de6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 11:56:46.852990 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG passlib.handlers.bcrypt [None req-3364db8e-f975-4b36-a6b7-62f3423f63a4 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80539) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} May 19 11:56:46.853449 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG passlib.handlers.bcrypt [None req-3364db8e-f975-4b36-a6b7-62f3423f63a4 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80539) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} May 19 11:56:46.900548 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.auth.core [None req-3364db8e-f975-4b36-a6b7-62f3423f63a4 None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 11:56:46.902190 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-3364db8e-f975-4b36-a6b7-62f3423f63a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 11:56:46.917410 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 7/12] 10.222.0.23 () {58 vars in 1023 bytes} [Thu May 19 11:56:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 11:56:46.940382 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45fc5d44-5593-4098-93d9-9ea257e45de6 None None] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:56:46.970451 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:56:47.007004 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 6/13] 10.222.0.23 () {58 vars in 1023 bytes} [Thu May 19 11:56:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 11:56:47.030782 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3364db8e-f975-4b36-a6b7-62f3423f63a4 None None] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:56:47.055988 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45fc5d44-5593-4098-93d9-9ea257e45de6 None None] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:56:47.094692 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c0c6a9-e9f4-4edc-9e3c-38237c7df360 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:56:47.096258 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c0c6a9-e9f4-4edc-9e3c-38237c7df360 None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:56:47.096258 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c0c6a9-e9f4-4edc-9e3c-38237c7df360 None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:56:47.096575 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c0c6a9-e9f4-4edc-9e3c-38237c7df360 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:56:47.172487 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c0c6a9-e9f4-4edc-9e3c-38237c7df360 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:56:47.174994 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c0c6a9-e9f4-4edc-9e3c-38237c7df360 None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:56:47.202721 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:56:47.242825 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3364db8e-f975-4b36-a6b7-62f3423f63a4 None None] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:56:47.245152 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:56:47.280414 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 8/14] 10.222.0.23 () {58 vars in 1282 bytes} [Thu May 19 11:56:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 19 11:56:47.282762 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e79177-2410-4f7e-816f-0e5a6e3e84ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:56:47.284423 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e79177-2410-4f7e-816f-0e5a6e3e84ed None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:56:47.284715 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e79177-2410-4f7e-816f-0e5a6e3e84ed None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:56:47.284996 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e79177-2410-4f7e-816f-0e5a6e3e84ed None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:56:47.286508 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c0c6a9-e9f4-4edc-9e3c-38237c7df360 None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:56:47.297372 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e79177-2410-4f7e-816f-0e5a6e3e84ed None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 11:56:47.309825 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application [None req-b3e79177-2410-4f7e-816f-0e5a6e3e84ed None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 11:56:47.309825 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 11:56:47.309825 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 11:56:47.309825 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 11:56:47.309825 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 11:56:47.309825 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 11:56:47.309825 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 11:56:47.309825 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 11:56:47.309825 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 11:56:47.309825 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 11:56:47.309825 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 11:56:47.309825 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 11:56:47.309825 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 11:56:47.309825 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 11:56:47.309825 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 11:56:47.309825 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 11:56:47.310960 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 11:56:47.310960 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 11:56:47.310960 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 11:56:47.310960 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 11:56:47.310960 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 11:56:47.310960 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 11:56:47.310960 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 11:56:47.310960 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 11:56:47.310960 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 11:56:47.310960 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.get_or_create( May 19 11:56:47.310960 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 11:56:47.310960 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application with Lock( May 19 11:56:47.310960 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 11:56:47.310960 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._enter() May 19 11:56:47.310960 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 11:56:47.310960 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 11:56:47.311984 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 11:56:47.311984 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.creator() May 19 11:56:47.311984 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 11:56:47.311984 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application created_value = creator( May 19 11:56:47.311984 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 11:56:47.311984 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 11:56:47.311984 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 11:56:47.311984 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 11:56:47.311984 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 11:56:47.311984 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 11:56:47.311984 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 11:56:47.311984 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application May 19 11:56:47.311984 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 7/15] 10.222.0.23 () {56 vars in 1096 bytes} [Thu May 19 11:56:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 11:56:47.316769 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e79177-2410-4f7e-816f-0e5a6e3e84ed None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:56:47.389949 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c0c6a9-e9f4-4edc-9e3c-38237c7df360 None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:56:47.413457 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:56:47.417721 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e79177-2410-4f7e-816f-0e5a6e3e84ed None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:56:47.425886 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f563e57c-4846-4e62-a800-95c2050b5105 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:56:47.426691 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f563e57c-4846-4e62-a800-95c2050b5105 None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:56:47.428260 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f563e57c-4846-4e62-a800-95c2050b5105 None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:56:47.428260 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f563e57c-4846-4e62-a800-95c2050b5105 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:56:47.437286 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f563e57c-4846-4e62-a800-95c2050b5105 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:56:47.438912 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f563e57c-4846-4e62-a800-95c2050b5105 None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:56:47.453456 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777d0594-322c-46a3-bc0d-cc9344cf5afc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:56:47.454144 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777d0594-322c-46a3-bc0d-cc9344cf5afc None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:56:47.454493 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777d0594-322c-46a3-bc0d-cc9344cf5afc None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:56:47.454780 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777d0594-322c-46a3-bc0d-cc9344cf5afc None admin] PATH_INFO: `/v3/roles` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:56:47.455633 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777d0594-322c-46a3-bc0d-cc9344cf5afc None admin] RBAC: Adding query filter params (name=admin) {{(pid=80539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 11:56:47.455974 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777d0594-322c-46a3-bc0d-cc9344cf5afc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:56:47.457403 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777d0594-322c-46a3-bc0d-cc9344cf5afc None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:56:47.461605 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application [None req-f563e57c-4846-4e62-a800-95c2050b5105 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 11:56:47.461605 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 11:56:47.461605 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 11:56:47.461605 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 11:56:47.461605 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 11:56:47.461605 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 11:56:47.461605 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 11:56:47.461605 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 11:56:47.461605 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 11:56:47.461605 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 11:56:47.461605 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 11:56:47.461605 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 11:56:47.461605 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 11:56:47.461605 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 11:56:47.461605 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 11:56:47.462433 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 11:56:47.462433 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 11:56:47.462433 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 11:56:47.462433 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 11:56:47.462433 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 11:56:47.462433 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 11:56:47.462433 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.get_or_create( May 19 11:56:47.462433 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 11:56:47.462433 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application with Lock( May 19 11:56:47.462433 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 11:56:47.462433 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._enter() May 19 11:56:47.462433 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 11:56:47.462433 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 11:56:47.462433 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 11:56:47.462433 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.creator() May 19 11:56:47.462433 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 11:56:47.462433 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application created_value = creator( May 19 11:56:47.463201 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 11:56:47.463201 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 11:56:47.463201 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 11:56:47.463201 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application May 19 11:56:47.463201 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 9/16] 10.222.0.23 () {56 vars in 1108 bytes} [Thu May 19 11:56:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 11:56:47.468757 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f563e57c-4846-4e62-a800-95c2050b5105 None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:56:47.469382 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 8/17] 10.222.0.23 () {56 vars in 1099 bytes} [Thu May 19 11:56:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:56:47.475397 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777d0594-322c-46a3-bc0d-cc9344cf5afc None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:56:47.503147 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:56:47.572371 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f563e57c-4846-4e62-a800-95c2050b5105 None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:56:47.575375 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777d0594-322c-46a3-bc0d-cc9344cf5afc None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:56:47.606663 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e36cdb-4b61-4e8c-b50b-4c376597028b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:56:47.607317 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e36cdb-4b61-4e8c-b50b-4c376597028b None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:56:47.607599 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e36cdb-4b61-4e8c-b50b-4c376597028b None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:56:47.607904 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e36cdb-4b61-4e8c-b50b-4c376597028b None admin] PATH_INFO: `/v3/domains` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:56:47.608623 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e36cdb-4b61-4e8c-b50b-4c376597028b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 11:56:47.608927 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e36cdb-4b61-4e8c-b50b-4c376597028b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:56:47.609745 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74f337f-bd17-4d51-850e-1f35d24b98ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:56:47.610981 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e36cdb-4b61-4e8c-b50b-4c376597028b None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:56:47.611382 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74f337f-bd17-4d51-850e-1f35d24b98ef None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:56:47.611738 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74f337f-bd17-4d51-850e-1f35d24b98ef None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:56:47.612105 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74f337f-bd17-4d51-850e-1f35d24b98ef None admin] PATH_INFO: `/v3/users/11855e565a88475189d279de9469c271` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:56:47.613882 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74f337f-bd17-4d51-850e-1f35d24b98ef None admin] RBAC: Authorizing `identity:get_user(user_id=11855e565a88475189d279de9469c271)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:56:47.615329 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74f337f-bd17-4d51-850e-1f35d24b98ef None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:56:47.616988 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 9/18] 10.222.0.23 () {56 vars in 1177 bytes} [Thu May 19 11:56:47 2022] GET /identity/v3/users/11855e565a88475189d279de9469c271 => generated 247 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 11:56:47.622430 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74f337f-bd17-4d51-850e-1f35d24b98ef None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:56:47.629240 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 10/19] 10.222.0.23 () {56 vars in 1109 bytes} [Thu May 19 11:56:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 11:56:47.730311 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:56:47.738017 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74f337f-bd17-4d51-850e-1f35d24b98ef None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:56:47.775583 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881fbf12-9210-4e6c-9c09-d544179645a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:56:47.776560 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881fbf12-9210-4e6c-9c09-d544179645a8 None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:56:47.776900 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881fbf12-9210-4e6c-9c09-d544179645a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:56:47.777233 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881fbf12-9210-4e6c-9c09-d544179645a8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:56:47.778751 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881fbf12-9210-4e6c-9c09-d544179645a8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:56:47.780313 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881fbf12-9210-4e6c-9c09-d544179645a8 None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:56:47.781976 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 10/20] 10.222.0.23 () {56 vars in 1108 bytes} [Thu May 19 11:56:47 2022] GET /identity/v3/domains/default => generated 199 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 11:56:47.788793 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e36cdb-4b61-4e8c-b50b-4c376597028b None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:56:47.824884 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:56:47.896072 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e36cdb-4b61-4e8c-b50b-4c376597028b None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:56:47.937497 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341c0ec6-91da-4c1d-a566-4c86f0f38148 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:56:47.938169 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341c0ec6-91da-4c1d-a566-4c86f0f38148 None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:56:47.938487 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341c0ec6-91da-4c1d-a566-4c86f0f38148 None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:56:47.939467 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341c0ec6-91da-4c1d-a566-4c86f0f38148 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:56:47.940371 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341c0ec6-91da-4c1d-a566-4c86f0f38148 None admin] RBAC: Adding query filter params (role.id=9db5671c65d3414ea2682afa2ccf1403, scope.domain.id=default, user.id=11855e565a88475189d279de9469c271) {{(pid=80540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 11:56:47.940689 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341c0ec6-91da-4c1d-a566-4c86f0f38148 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:56:47.942194 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341c0ec6-91da-4c1d-a566-4c86f0f38148 None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:56:47.953820 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 11/21] 10.222.0.23 () {56 vars in 1322 bytes} [Thu May 19 11:56:47 2022] GET /identity/v3/role_assignments?user.id=11855e565a88475189d279de9469c271&scope.domain.id=default&role.id=9db5671c65d3414ea2682afa2ccf1403 => generated 233 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:56:50.554948 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bd01f0-4a19-4228-8849-be0a65bca1e3 None None] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:56:50.555990 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bd01f0-4a19-4228-8849-be0a65bca1e3 None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:56:50.556492 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bd01f0-4a19-4228-8849-be0a65bca1e3 None None] PATH_INFO: `/` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:56:50.558061 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 11/22] 10.222.0.23 () {54 vars in 926 bytes} [Thu May 19 11:56:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 11:56:50.563522 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7099ea27-66d5-40d7-b4d7-6464e2d6f4dc None None] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:56:50.564346 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7099ea27-66d5-40d7-b4d7-6464e2d6f4dc None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:56:50.564833 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7099ea27-66d5-40d7-b4d7-6464e2d6f4dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:56:50.588637 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.auth.core [None req-7099ea27-66d5-40d7-b4d7-6464e2d6f4dc None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 11:56:50.590284 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-7099ea27-66d5-40d7-b4d7-6464e2d6f4dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 11:56:50.686975 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 12/23] 10.222.0.23 () {58 vars in 1023 bytes} [Thu May 19 11:56:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 11:56:50.737516 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65bd01f0-4a19-4228-8849-be0a65bca1e3 None None] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:56:50.834529 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6f9104-19de-4163-a14f-359c79845d5f None None] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:56:50.835176 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6f9104-19de-4163-a14f-359c79845d5f None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:56:50.835671 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6f9104-19de-4163-a14f-359c79845d5f None None] PATH_INFO: `/` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:56:50.837249 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 13/24] 10.222.0.23 () {54 vars in 926 bytes} [Thu May 19 11:56:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 11:56:50.842285 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3501b899-2e6b-4f38-b27e-fdf4658b79f1 None None] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:56:50.842847 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3501b899-2e6b-4f38-b27e-fdf4658b79f1 None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:56:50.843321 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3501b899-2e6b-4f38-b27e-fdf4658b79f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:56:50.868267 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.auth.core [None req-3501b899-2e6b-4f38-b27e-fdf4658b79f1 None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 11:56:50.871383 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-3501b899-2e6b-4f38-b27e-fdf4658b79f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 11:56:50.918876 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65bd01f0-4a19-4228-8849-be0a65bca1e3 None None] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:56:50.936879 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:56:50.967963 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111e520d-b702-4b2d-b9bb-955bd9731476 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:56:50.969593 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111e520d-b702-4b2d-b9bb-955bd9731476 None admin] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:56:50.969593 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111e520d-b702-4b2d-b9bb-955bd9731476 None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:56:50.969593 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111e520d-b702-4b2d-b9bb-955bd9731476 None admin] PATH_INFO: `/v3/roles` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:56:50.970793 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111e520d-b702-4b2d-b9bb-955bd9731476 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:56:50.972751 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111e520d-b702-4b2d-b9bb-955bd9731476 None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:56:51.011228 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 14/25] 10.222.0.23 () {58 vars in 1023 bytes} [Thu May 19 11:56:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 11:56:51.012836 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d448721-89e1-497e-b931-75427b393b61 None None] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:56:51.013270 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d448721-89e1-497e-b931-75427b393b61 None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:56:51.013688 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d448721-89e1-497e-b931-75427b393b61 None None] PATH_INFO: `/` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:56:51.015311 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 15/26] 10.222.0.23 () {54 vars in 926 bytes} [Thu May 19 11:56:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 11:56:51.021164 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1542c3-f903-4369-91a8-57ecb874d0e8 None None] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:56:51.021637 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1542c3-f903-4369-91a8-57ecb874d0e8 None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:56:51.022054 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1542c3-f903-4369-91a8-57ecb874d0e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:56:51.044924 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.auth.core [None req-dd1542c3-f903-4369-91a8-57ecb874d0e8 None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 11:56:51.046386 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-dd1542c3-f903-4369-91a8-57ecb874d0e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 11:56:51.149090 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 16/27] 10.222.0.23 () {58 vars in 1023 bytes} [Thu May 19 11:56:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 11:56:51.150721 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1542c3-f903-4369-91a8-57ecb874d0e8 None None] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:56:51.252719 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:56:51.264745 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1542c3-f903-4369-91a8-57ecb874d0e8 None None] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:56:51.288944 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 12/28] 10.222.0.23 () {60 vars in 1131 bytes} [Thu May 19 11:56:50 2022] POST /identity/v3/roles => generated 219 bytes in 552 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 11:56:51.289979 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111e520d-b702-4b2d-b9bb-955bd9731476 None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:56:51.302340 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d030c4-9c57-42e4-b02e-e87f44ed9a27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:56:51.303257 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d030c4-9c57-42e4-b02e-e87f44ed9a27 None admin] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:56:51.303730 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d030c4-9c57-42e4-b02e-e87f44ed9a27 None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:56:51.304168 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d030c4-9c57-42e4-b02e-e87f44ed9a27 None admin] PATH_INFO: `/v3/roles` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:56:51.305116 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d030c4-9c57-42e4-b02e-e87f44ed9a27 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:56:51.306734 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d030c4-9c57-42e4-b02e-e87f44ed9a27 None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:56:51.355261 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:56:51.512994 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111e520d-b702-4b2d-b9bb-955bd9731476 None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:56:51.533585 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 17/29] 10.222.0.23 () {60 vars in 1131 bytes} [Thu May 19 11:56:51 2022] POST /identity/v3/roles => generated 225 bytes in 384 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 11:56:51.535400 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063039e6-89e2-4d57-9c8e-6a0cabcb7a4b None None] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:56:51.535959 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063039e6-89e2-4d57-9c8e-6a0cabcb7a4b None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:56:51.536383 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063039e6-89e2-4d57-9c8e-6a0cabcb7a4b None None] PATH_INFO: `/` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:56:51.537917 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 18/30] 10.222.0.23 () {54 vars in 926 bytes} [Thu May 19 11:56:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 11:56:51.543866 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a299a1-e7c6-4463-a5ba-0aadb98722d8 None None] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:56:51.544391 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a299a1-e7c6-4463-a5ba-0aadb98722d8 None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:56:51.544818 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a299a1-e7c6-4463-a5ba-0aadb98722d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:56:51.566200 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.auth.core [None req-46a299a1-e7c6-4463-a5ba-0aadb98722d8 None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 11:56:51.568647 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-46a299a1-e7c6-4463-a5ba-0aadb98722d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 11:56:51.616565 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e80fa19-421b-4e86-bf6b-a0a6fd9d2534 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:56:51.617404 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e80fa19-421b-4e86-bf6b-a0a6fd9d2534 None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:56:51.617698 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e80fa19-421b-4e86-bf6b-a0a6fd9d2534 None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:56:51.617985 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e80fa19-421b-4e86-bf6b-a0a6fd9d2534 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:56:51.647895 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e80fa19-421b-4e86-bf6b-a0a6fd9d2534 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:56:51.649010 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e80fa19-421b-4e86-bf6b-a0a6fd9d2534 None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:56:51.673791 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 19/31] 10.222.0.23 () {58 vars in 1023 bytes} [Thu May 19 11:56:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 11:56:51.679835 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application [None req-8e80fa19-421b-4e86-bf6b-a0a6fd9d2534 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 11:56:51.679835 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 11:56:51.679835 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 11:56:51.679835 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 11:56:51.679835 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 11:56:51.679835 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 11:56:51.679835 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 11:56:51.679835 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 11:56:51.679835 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 11:56:51.679835 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 11:56:51.679835 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 11:56:51.679835 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 11:56:51.679835 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 11:56:51.679835 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 11:56:51.679835 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 11:56:51.680670 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 11:56:51.680670 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 11:56:51.680670 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 11:56:51.680670 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 11:56:51.680670 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 11:56:51.680670 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 11:56:51.680670 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.get_or_create( May 19 11:56:51.680670 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 11:56:51.680670 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application with Lock( May 19 11:56:51.680670 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 11:56:51.680670 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._enter() May 19 11:56:51.680670 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 11:56:51.680670 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 11:56:51.680670 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 11:56:51.680670 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.creator() May 19 11:56:51.680670 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 11:56:51.680670 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application created_value = creator( May 19 11:56:51.681454 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 11:56:51.681454 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 11:56:51.681454 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 11:56:51.681454 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application May 19 11:56:51.681454 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 13/32] 10.222.0.23 () {56 vars in 1108 bytes} [Thu May 19 11:56:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 388 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 11:56:51.682924 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e510a8-298f-490d-9882-c3baac50d16e None None] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:56:51.683388 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e510a8-298f-490d-9882-c3baac50d16e None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:56:51.683867 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e510a8-298f-490d-9882-c3baac50d16e None None] PATH_INFO: `/` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:56:51.684202 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e80fa19-421b-4e86-bf6b-a0a6fd9d2534 None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:56:51.685780 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 20/33] 10.222.0.23 () {54 vars in 926 bytes} [Thu May 19 11:56:51 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 11:56:51.694790 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ce67fc-5533-4d10-8881-53c8b93eea87 None None] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:56:51.695659 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ce67fc-5533-4d10-8881-53c8b93eea87 None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:56:51.696127 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ce67fc-5533-4d10-8881-53c8b93eea87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:56:51.725693 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.auth.core [None req-00ce67fc-5533-4d10-8881-53c8b93eea87 None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 11:56:51.727245 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-00ce67fc-5533-4d10-8881-53c8b93eea87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 11:56:51.792203 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:56:51.817611 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 21/34] 10.222.0.23 () {58 vars in 1023 bytes} [Thu May 19 11:56:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 11:56:51.818819 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ce67fc-5533-4d10-8881-53c8b93eea87 None None] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:56:51.822494 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e80fa19-421b-4e86-bf6b-a0a6fd9d2534 None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:56:51.836291 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:56:51.867261 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8e6276-532d-4cd6-8a92-646b2836587d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:56:51.868422 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8e6276-532d-4cd6-8a92-646b2836587d None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:56:51.868996 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8e6276-532d-4cd6-8a92-646b2836587d None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:56:51.869535 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8e6276-532d-4cd6-8a92-646b2836587d None admin] PATH_INFO: `/v3/domains` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:56:51.870599 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8e6276-532d-4cd6-8a92-646b2836587d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 11:56:51.871162 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8e6276-532d-4cd6-8a92-646b2836587d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:56:51.873026 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8e6276-532d-4cd6-8a92-646b2836587d None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:56:51.898449 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 14/35] 10.222.0.23 () {56 vars in 1109 bytes} [Thu May 19 11:56:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:56:51.899168 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8e6276-532d-4cd6-8a92-646b2836587d None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:56:51.963623 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ce67fc-5533-4d10-8881-53c8b93eea87 None None] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:56:51.997762 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d45c8f7-a3ff-4f6f-ab45-578ef61fec04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:56:51.998560 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d45c8f7-a3ff-4f6f-ab45-578ef61fec04 None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:56:51.998850 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d45c8f7-a3ff-4f6f-ab45-578ef61fec04 None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:56:51.999133 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d45c8f7-a3ff-4f6f-ab45-578ef61fec04 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:56:52.000330 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d45c8f7-a3ff-4f6f-ab45-578ef61fec04 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:56:52.002458 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d45c8f7-a3ff-4f6f-ab45-578ef61fec04 None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:56:52.004280 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 22/36] 10.222.0.23 () {56 vars in 1108 bytes} [Thu May 19 11:56:51 2022] GET /identity/v3/domains/default => generated 199 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:56:52.004921 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d45c8f7-a3ff-4f6f-ab45-578ef61fec04 None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:56:52.018101 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8e6276-532d-4cd6-8a92-646b2836587d None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:56:52.051780 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbe257e-2716-4430-9c3f-ce1a198d461e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:56:52.052531 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbe257e-2716-4430-9c3f-ce1a198d461e None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:56:52.052823 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbe257e-2716-4430-9c3f-ce1a198d461e None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:56:52.053108 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbe257e-2716-4430-9c3f-ce1a198d461e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:56:52.055000 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:56:52.064057 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fbe257e-2716-4430-9c3f-ce1a198d461e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 11:56:52.071992 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application [None req-1fbe257e-2716-4430-9c3f-ce1a198d461e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 11:56:52.071992 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 11:56:52.071992 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 11:56:52.071992 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 11:56:52.071992 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 11:56:52.071992 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 11:56:52.071992 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 11:56:52.071992 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 11:56:52.071992 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 11:56:52.071992 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 11:56:52.071992 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 11:56:52.071992 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 11:56:52.071992 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 11:56:52.071992 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 11:56:52.071992 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 11:56:52.071992 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 11:56:52.072999 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 11:56:52.072999 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 11:56:52.072999 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 11:56:52.072999 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 11:56:52.072999 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 11:56:52.072999 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 11:56:52.072999 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 11:56:52.072999 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 11:56:52.072999 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 11:56:52.072999 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.get_or_create( May 19 11:56:52.072999 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 11:56:52.072999 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application with Lock( May 19 11:56:52.072999 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 11:56:52.072999 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._enter() May 19 11:56:52.072999 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 11:56:52.072999 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 11:56:52.073914 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 11:56:52.073914 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.creator() May 19 11:56:52.073914 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 11:56:52.073914 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application created_value = creator( May 19 11:56:52.073914 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 11:56:52.073914 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 11:56:52.073914 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 11:56:52.073914 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 11:56:52.073914 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 11:56:52.073914 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 11:56:52.073914 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 11:56:52.073914 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application May 19 11:56:52.073914 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 15/37] 10.222.0.23 () {56 vars in 1096 bytes} [Thu May 19 11:56:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 11:56:52.074715 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175d9ffd-b7f2-4c12-b7ec-950e889cd697 None None] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:56:52.074788 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175d9ffd-b7f2-4c12-b7ec-950e889cd697 None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:56:52.075121 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175d9ffd-b7f2-4c12-b7ec-950e889cd697 None None] PATH_INFO: `/` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:56:52.076497 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 16/38] 10.222.0.23 () {54 vars in 926 bytes} [Thu May 19 11:56:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 11:56:52.077200 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175d9ffd-b7f2-4c12-b7ec-950e889cd697 None None] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:56:52.091797 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:56:52.114861 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d45c8f7-a3ff-4f6f-ab45-578ef61fec04 None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:56:52.149602 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b586bd2-ea3a-4775-8f01-920dead85a5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:56:52.150272 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b586bd2-ea3a-4775-8f01-920dead85a5a None admin] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:56:52.150574 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b586bd2-ea3a-4775-8f01-920dead85a5a None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:56:52.150861 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b586bd2-ea3a-4775-8f01-920dead85a5a None admin] PATH_INFO: `/v3/projects` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:56:52.151580 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b586bd2-ea3a-4775-8f01-920dead85a5a None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:56:52.153101 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b586bd2-ea3a-4775-8f01-920dead85a5a None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:56:52.181159 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175d9ffd-b7f2-4c12-b7ec-950e889cd697 None None] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:56:52.222647 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372e56e6-48ca-42ff-b4e0-3e15b8aaf3af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:56:52.223312 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372e56e6-48ca-42ff-b4e0-3e15b8aaf3af None admin] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:56:52.223599 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372e56e6-48ca-42ff-b4e0-3e15b8aaf3af None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:56:52.223909 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372e56e6-48ca-42ff-b4e0-3e15b8aaf3af None admin] PATH_INFO: `/v3/projects` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:56:52.224620 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372e56e6-48ca-42ff-b4e0-3e15b8aaf3af None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:56:52.226089 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372e56e6-48ca-42ff-b4e0-3e15b8aaf3af None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:56:52.259368 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 23/39] 10.222.0.23 () {60 vars in 1141 bytes} [Thu May 19 11:56:52 2022] POST /identity/v3/projects => generated 301 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 11:56:52.260351 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b586bd2-ea3a-4775-8f01-920dead85a5a None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:56:52.368398 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 17/40] 10.222.0.23 () {60 vars in 1141 bytes} [Thu May 19 11:56:52 2022] POST /identity/v3/projects => generated 312 bytes in 291 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 11:56:52.369840 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b8dba2-dce2-42d2-81a9-71eb9f757ba7 None None] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:56:52.370199 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b8dba2-dce2-42d2-81a9-71eb9f757ba7 None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:56:52.370545 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b8dba2-dce2-42d2-81a9-71eb9f757ba7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:56:52.389253 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b586bd2-ea3a-4775-8f01-920dead85a5a None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:56:52.396494 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.auth.core [None req-b2b8dba2-dce2-42d2-81a9-71eb9f757ba7 None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 11:56:52.398701 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-b2b8dba2-dce2-42d2-81a9-71eb9f757ba7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 11:56:52.436387 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:56:52.448650 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8778df2a-a3cf-49f0-98ae-85198cc43d86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:56:52.449737 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8778df2a-a3cf-49f0-98ae-85198cc43d86 None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:56:52.450200 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8778df2a-a3cf-49f0-98ae-85198cc43d86 None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:56:52.453521 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8778df2a-a3cf-49f0-98ae-85198cc43d86 None admin] PATH_INFO: `/v3/roles` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:56:52.453521 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8778df2a-a3cf-49f0-98ae-85198cc43d86 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 11:56:52.453521 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8778df2a-a3cf-49f0-98ae-85198cc43d86 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:56:52.454621 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8778df2a-a3cf-49f0-98ae-85198cc43d86 None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:56:52.471353 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 24/41] 10.222.0.23 () {56 vars in 1099 bytes} [Thu May 19 11:56:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:56:52.474433 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:56:52.477685 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8778df2a-a3cf-49f0-98ae-85198cc43d86 None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:56:52.511408 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 18/42] 10.222.0.23 () {58 vars in 1023 bytes} [Thu May 19 11:56:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 11:56:52.535816 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b8dba2-dce2-42d2-81a9-71eb9f757ba7 None None] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:56:52.615036 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8778df2a-a3cf-49f0-98ae-85198cc43d86 None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:56:52.655637 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3260b4be-9d85-41dc-ac41-07df9eedeeb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:56:52.656478 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3260b4be-9d85-41dc-ac41-07df9eedeeb3 None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:56:52.656770 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3260b4be-9d85-41dc-ac41-07df9eedeeb3 None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:56:52.656871 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b8dba2-dce2-42d2-81a9-71eb9f757ba7 None None] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:56:52.657376 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3260b4be-9d85-41dc-ac41-07df9eedeeb3 None admin] PATH_INFO: `/v3/users/11855e565a88475189d279de9469c271` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:56:52.658649 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3260b4be-9d85-41dc-ac41-07df9eedeeb3 None admin] RBAC: Authorizing `identity:get_user(user_id=11855e565a88475189d279de9469c271)` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:56:52.660168 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3260b4be-9d85-41dc-ac41-07df9eedeeb3 None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:56:52.662222 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 25/43] 10.222.0.23 () {56 vars in 1177 bytes} [Thu May 19 11:56:52 2022] GET /identity/v3/users/11855e565a88475189d279de9469c271 => generated 247 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:56:52.669563 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3260b4be-9d85-41dc-ac41-07df9eedeeb3 None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:56:52.702944 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9309b7-0e26-4e98-b6a5-efb00beb14f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:56:52.703634 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9309b7-0e26-4e98-b6a5-efb00beb14f4 None admin] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:56:52.704619 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9309b7-0e26-4e98-b6a5-efb00beb14f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:56:52.705042 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9309b7-0e26-4e98-b6a5-efb00beb14f4 None admin] PATH_INFO: `/v3/roles` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:56:52.705816 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9309b7-0e26-4e98-b6a5-efb00beb14f4 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:56:52.707225 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9309b7-0e26-4e98-b6a5-efb00beb14f4 None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:56:52.732754 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:56:52.775796 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3260b4be-9d85-41dc-ac41-07df9eedeeb3 None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:56:52.807287 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 19/44] 10.222.0.23 () {60 vars in 1131 bytes} [Thu May 19 11:56:52 2022] POST /identity/v3/roles => generated 223 bytes in 272 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 11:56:52.819199 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9b1cc3-475c-4bf7-96ff-369a7f00895b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:56:52.820391 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9b1cc3-475c-4bf7-96ff-369a7f00895b None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:56:52.820687 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9b1cc3-475c-4bf7-96ff-369a7f00895b None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:56:52.820973 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9b1cc3-475c-4bf7-96ff-369a7f00895b None admin] PATH_INFO: `/v3/domains/default` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:56:52.822332 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9b1cc3-475c-4bf7-96ff-369a7f00895b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:56:52.824128 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9b1cc3-475c-4bf7-96ff-369a7f00895b None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:56:52.826149 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 26/45] 10.222.0.23 () {56 vars in 1108 bytes} [Thu May 19 11:56:52 2022] GET /identity/v3/domains/default => generated 199 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 11:56:52.832707 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9309b7-0e26-4e98-b6a5-efb00beb14f4 None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:56:52.872837 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:56:52.954667 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9309b7-0e26-4e98-b6a5-efb00beb14f4 None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:56:52.993479 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5eeb2f0-79fb-4be3-a729-6870e20b424a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:56:52.994373 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5eeb2f0-79fb-4be3-a729-6870e20b424a None admin] REQUEST_METHOD: `PUT` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:56:52.994658 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5eeb2f0-79fb-4be3-a729-6870e20b424a None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:56:52.994944 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5eeb2f0-79fb-4be3-a729-6870e20b424a None admin] PATH_INFO: `/v3/domains/default/users/11855e565a88475189d279de9469c271/roles/9db5671c65d3414ea2682afa2ccf1403` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:56:52.997084 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5eeb2f0-79fb-4be3-a729-6870e20b424a None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=11855e565a88475189d279de9469c271, role_id=9db5671c65d3414ea2682afa2ccf1403)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:56:52.998573 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5eeb2f0-79fb-4be3-a729-6870e20b424a None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:56:53.075222 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 20/46] 10.222.0.23 () {58 vars in 1361 bytes} [Thu May 19 11:56:52 2022] PUT /identity/v3/domains/default/users/11855e565a88475189d279de9469c271/roles/9db5671c65d3414ea2682afa2ccf1403 => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 11:56:54.674148 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e7acda-8769-4b35-b76d-1cc514195cb9 None None] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:56:54.674711 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e7acda-8769-4b35-b76d-1cc514195cb9 None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:56:54.674829 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e7acda-8769-4b35-b76d-1cc514195cb9 None None] PATH_INFO: `/` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:56:54.676383 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 27/47] 10.222.0.23 () {54 vars in 926 bytes} [Thu May 19 11:56:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 11:56:54.681845 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05085e64-32b5-4d92-b966-e32b89f63f02 None None] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:56:54.682265 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05085e64-32b5-4d92-b966-e32b89f63f02 None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:56:54.682398 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05085e64-32b5-4d92-b966-e32b89f63f02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:56:54.699864 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.auth.core [None req-05085e64-32b5-4d92-b966-e32b89f63f02 None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 11:56:54.701069 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-05085e64-32b5-4d92-b966-e32b89f63f02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 11:56:54.771862 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 21/48] 10.222.0.23 () {58 vars in 1023 bytes} [Thu May 19 11:56:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 11:56:54.794652 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e7acda-8769-4b35-b76d-1cc514195cb9 None None] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:56:54.905375 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e7acda-8769-4b35-b76d-1cc514195cb9 None None] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:56:54.931936 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:56:54.941269 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbb96cf-f4e3-4483-be3f-38f99aafd763 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:56:54.942077 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbb96cf-f4e3-4483-be3f-38f99aafd763 None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:56:54.942881 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbb96cf-f4e3-4483-be3f-38f99aafd763 None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:56:54.942881 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbb96cf-f4e3-4483-be3f-38f99aafd763 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:56:54.943891 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bbb96cf-f4e3-4483-be3f-38f99aafd763 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:56:54.945642 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bbb96cf-f4e3-4483-be3f-38f99aafd763 None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:56:54.947436 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 28/49] 10.222.0.23 () {56 vars in 1108 bytes} [Thu May 19 11:56:54 2022] GET /identity/v3/domains/default => generated 199 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:56:54.952838 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05085e64-32b5-4d92-b966-e32b89f63f02 None None] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:56:54.981231 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:56:55.054751 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05085e64-32b5-4d92-b966-e32b89f63f02 None None] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:56:55.092495 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2e00c2-f1bb-4b26-b2f8-96ef0712b997 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:56:55.093188 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2e00c2-f1bb-4b26-b2f8-96ef0712b997 None admin] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:56:55.093481 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2e00c2-f1bb-4b26-b2f8-96ef0712b997 None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:56:55.093765 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2e00c2-f1bb-4b26-b2f8-96ef0712b997 None admin] PATH_INFO: `/v3/projects` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:56:55.094709 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2e00c2-f1bb-4b26-b2f8-96ef0712b997 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:56:55.096414 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2e00c2-f1bb-4b26-b2f8-96ef0712b997 None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:56:55.173949 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 22/50] 10.222.0.23 () {60 vars in 1140 bytes} [Thu May 19 11:56:54 2022] POST /identity/v3/projects => generated 298 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 11:56:55.275508 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f878faa4-e7bb-407f-abdc-c414401c5c80 None None] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:56:55.275749 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f878faa4-e7bb-407f-abdc-c414401c5c80 None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:56:55.276097 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f878faa4-e7bb-407f-abdc-c414401c5c80 None None] PATH_INFO: `/` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:56:55.277564 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 29/51] 10.222.0.23 () {54 vars in 926 bytes} [Thu May 19 11:56:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 11:56:55.283471 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba6eaa6-037c-447a-8518-b3d9ffe79da9 None None] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:56:55.283758 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba6eaa6-037c-447a-8518-b3d9ffe79da9 None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:56:55.284068 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba6eaa6-037c-447a-8518-b3d9ffe79da9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:56:55.303111 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.auth.core [None req-eba6eaa6-037c-447a-8518-b3d9ffe79da9 None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 11:56:55.304368 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-eba6eaa6-037c-447a-8518-b3d9ffe79da9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 11:56:55.377523 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 23/52] 10.222.0.23 () {58 vars in 1023 bytes} [Thu May 19 11:56:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 11:56:55.400025 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f878faa4-e7bb-407f-abdc-c414401c5c80 None None] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:56:55.503619 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f878faa4-e7bb-407f-abdc-c414401c5c80 None None] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:56:55.537764 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1708bb5-e3d2-46c2-956c-b61921c005a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:56:55.538503 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1708bb5-e3d2-46c2-956c-b61921c005a1 None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:56:55.538796 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1708bb5-e3d2-46c2-956c-b61921c005a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:56:55.539082 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1708bb5-e3d2-46c2-956c-b61921c005a1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:56:55.551175 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1708bb5-e3d2-46c2-956c-b61921c005a1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 11:56:55.560163 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application [None req-c1708bb5-e3d2-46c2-956c-b61921c005a1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 11:56:55.560163 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 11:56:55.560163 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 11:56:55.560163 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 11:56:55.560163 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 11:56:55.560163 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 11:56:55.560163 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 11:56:55.560163 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 11:56:55.560163 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 11:56:55.560163 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 11:56:55.560163 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 11:56:55.560163 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 11:56:55.560163 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 11:56:55.560163 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 11:56:55.560163 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 11:56:55.560163 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 11:56:55.560992 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 11:56:55.560992 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 11:56:55.560992 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 11:56:55.560992 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 11:56:55.560992 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 11:56:55.560992 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 11:56:55.560992 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 11:56:55.560992 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 11:56:55.560992 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 11:56:55.560992 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.get_or_create( May 19 11:56:55.560992 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 11:56:55.560992 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application with Lock( May 19 11:56:55.560992 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 11:56:55.560992 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._enter() May 19 11:56:55.560992 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 11:56:55.560992 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 11:56:55.561781 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 11:56:55.561781 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.creator() May 19 11:56:55.561781 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 11:56:55.561781 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application created_value = creator( May 19 11:56:55.561781 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 11:56:55.561781 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 11:56:55.561781 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 11:56:55.561781 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 11:56:55.561781 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 11:56:55.561781 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 11:56:55.561781 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 11:56:55.561781 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application May 19 11:56:55.561781 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 30/53] 10.222.0.23 () {56 vars in 1096 bytes} [Thu May 19 11:56:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 11:56:55.567929 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba6eaa6-037c-447a-8518-b3d9ffe79da9 None None] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:56:55.578626 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:56:55.686500 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba6eaa6-037c-447a-8518-b3d9ffe79da9 None None] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:56:55.722368 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5f0dc6-085a-4f7e-83aa-6a3566872523 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:56:55.723045 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5f0dc6-085a-4f7e-83aa-6a3566872523 None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:56:55.723377 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5f0dc6-085a-4f7e-83aa-6a3566872523 None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:56:55.723661 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5f0dc6-085a-4f7e-83aa-6a3566872523 None admin] PATH_INFO: `/v3/roles` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:56:55.724449 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5f0dc6-085a-4f7e-83aa-6a3566872523 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 11:56:55.724767 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5f0dc6-085a-4f7e-83aa-6a3566872523 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:56:55.726149 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5f0dc6-085a-4f7e-83aa-6a3566872523 None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:56:55.734771 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 24/54] 10.222.0.23 () {56 vars in 1099 bytes} [Thu May 19 11:56:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:56:55.741617 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1708bb5-e3d2-46c2-956c-b61921c005a1 None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:56:55.796842 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:56:55.842200 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1708bb5-e3d2-46c2-956c-b61921c005a1 None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:56:55.875412 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef872b91-dec0-4f83-8fe1-0bda6ef32618 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:56:55.877148 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef872b91-dec0-4f83-8fe1-0bda6ef32618 None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:56:55.877435 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef872b91-dec0-4f83-8fe1-0bda6ef32618 None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:56:55.877712 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef872b91-dec0-4f83-8fe1-0bda6ef32618 None admin] PATH_INFO: `/v3/users/11855e565a88475189d279de9469c271` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:56:55.878906 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef872b91-dec0-4f83-8fe1-0bda6ef32618 None admin] RBAC: Authorizing `identity:get_user(user_id=11855e565a88475189d279de9469c271)` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:56:55.880416 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef872b91-dec0-4f83-8fe1-0bda6ef32618 None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:56:55.882260 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 31/55] 10.222.0.23 () {56 vars in 1177 bytes} [Thu May 19 11:56:55 2022] GET /identity/v3/users/11855e565a88475189d279de9469c271 => generated 247 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 11:56:55.887458 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5f0dc6-085a-4f7e-83aa-6a3566872523 None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:56:55.962995 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:56:56.000354 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5f0dc6-085a-4f7e-83aa-6a3566872523 None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:56:56.036818 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02826320-9998-42da-afab-7f543ed94daa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:56:56.037614 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02826320-9998-42da-afab-7f543ed94daa None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:56:56.037905 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02826320-9998-42da-afab-7f543ed94daa None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:56:56.038187 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02826320-9998-42da-afab-7f543ed94daa None admin] PATH_INFO: `/v3/domains/default` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:56:56.039382 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02826320-9998-42da-afab-7f543ed94daa None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:56:56.040888 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02826320-9998-42da-afab-7f543ed94daa None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:56:56.042530 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 25/56] 10.222.0.23 () {56 vars in 1108 bytes} [Thu May 19 11:56:55 2022] GET /identity/v3/domains/default => generated 199 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:56:56.048905 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef872b91-dec0-4f83-8fe1-0bda6ef32618 None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:56:56.147439 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef872b91-dec0-4f83-8fe1-0bda6ef32618 None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:56:56.170327 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:56:56.182481 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2705e8e-3656-4758-b927-57fcfb72e388 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:56:56.183159 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2705e8e-3656-4758-b927-57fcfb72e388 None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:56:56.183452 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2705e8e-3656-4758-b927-57fcfb72e388 None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:56:56.183736 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2705e8e-3656-4758-b927-57fcfb72e388 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:56:56.184656 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2705e8e-3656-4758-b927-57fcfb72e388 None admin] RBAC: Adding query filter params (role.id=9db5671c65d3414ea2682afa2ccf1403, scope.domain.id=default, user.id=11855e565a88475189d279de9469c271) {{(pid=80540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 11:56:56.184981 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2705e8e-3656-4758-b927-57fcfb72e388 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:56:56.186488 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2705e8e-3656-4758-b927-57fcfb72e388 None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:56:56.193996 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 32/57] 10.222.0.23 () {56 vars in 1322 bytes} [Thu May 19 11:56:56 2022] GET /identity/v3/role_assignments?user.id=11855e565a88475189d279de9469c271&scope.domain.id=default&role.id=9db5671c65d3414ea2682afa2ccf1403 => generated 530 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:56:57.470771 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75506d7a-c6c5-4554-abcb-d15f24b6be23 None None] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:56:57.471301 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75506d7a-c6c5-4554-abcb-d15f24b6be23 None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:56:57.471387 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75506d7a-c6c5-4554-abcb-d15f24b6be23 None None] PATH_INFO: `/` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:56:57.472825 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 26/58] 10.222.0.23 () {54 vars in 926 bytes} [Thu May 19 11:56:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 11:56:57.478616 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb63c87f-428f-4be4-bd02-9c4939001671 None None] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:56:57.479027 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb63c87f-428f-4be4-bd02-9c4939001671 None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:56:57.479190 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb63c87f-428f-4be4-bd02-9c4939001671 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:56:57.496233 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.auth.core [None req-bb63c87f-428f-4be4-bd02-9c4939001671 None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 11:56:57.497518 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-bb63c87f-428f-4be4-bd02-9c4939001671 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 11:56:57.565654 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 33/59] 10.222.0.23 () {58 vars in 1023 bytes} [Thu May 19 11:56:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 11:56:57.590928 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75506d7a-c6c5-4554-abcb-d15f24b6be23 None None] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:56:57.691736 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75506d7a-c6c5-4554-abcb-d15f24b6be23 None None] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:56:57.726409 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3644e75a-efb1-46d5-ba64-c3959bd54caf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:56:57.727133 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3644e75a-efb1-46d5-ba64-c3959bd54caf None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:56:57.727424 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3644e75a-efb1-46d5-ba64-c3959bd54caf None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:56:57.727706 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3644e75a-efb1-46d5-ba64-c3959bd54caf None admin] PATH_INFO: `/v3/domains/default` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:56:57.728910 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3644e75a-efb1-46d5-ba64-c3959bd54caf None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:56:57.730381 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3644e75a-efb1-46d5-ba64-c3959bd54caf None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:56:57.732017 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 27/60] 10.222.0.23 () {56 vars in 1108 bytes} [Thu May 19 11:56:57 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:56:57.737834 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb63c87f-428f-4be4-bd02-9c4939001671 None None] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:56:57.840844 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb63c87f-428f-4be4-bd02-9c4939001671 None None] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:56:57.857173 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:56:57.876249 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e470b462-b910-43c6-962d-35c22dcd6598 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:56:57.876917 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e470b462-b910-43c6-962d-35c22dcd6598 None admin] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:56:57.877205 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e470b462-b910-43c6-962d-35c22dcd6598 None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:56:57.877486 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e470b462-b910-43c6-962d-35c22dcd6598 None admin] PATH_INFO: `/v3/users` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:56:57.878194 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e470b462-b910-43c6-962d-35c22dcd6598 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:56:57.879680 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e470b462-b910-43c6-962d-35c22dcd6598 None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:56:57.972139 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 34/61] 10.222.0.23 () {60 vars in 1132 bytes} [Thu May 19 11:56:57 2022] POST /identity/v3/users => generated 275 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 11:57:01.510063 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec012d4-3d77-4e1f-a21f-c5e1be92209a None None] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:01.511103 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec012d4-3d77-4e1f-a21f-c5e1be92209a None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:01.511722 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec012d4-3d77-4e1f-a21f-c5e1be92209a None None] PATH_INFO: `/` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:01.513404 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 28/62] 10.222.0.23 () {54 vars in 926 bytes} [Thu May 19 11:57:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 11:57:01.519476 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdc2823-00de-4c20-b35a-bf9857fdab70 None None] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:01.520467 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdc2823-00de-4c20-b35a-bf9857fdab70 None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:01.520956 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdc2823-00de-4c20-b35a-bf9857fdab70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:01.549746 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.auth.core [None req-7cdc2823-00de-4c20-b35a-bf9857fdab70 None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 11:57:01.551585 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-7cdc2823-00de-4c20-b35a-bf9857fdab70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 11:57:01.615459 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c96eb9-dd25-4996-b24a-aea5d573dc20 None None] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:01.616200 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c96eb9-dd25-4996-b24a-aea5d573dc20 None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:01.616688 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c96eb9-dd25-4996-b24a-aea5d573dc20 None None] PATH_INFO: `/` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:01.618227 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 29/63] 10.222.0.23 () {54 vars in 926 bytes} [Thu May 19 11:57:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 11:57:01.623811 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec9a5e0-b133-4598-941c-540dbcb8d5d9 None None] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:01.624381 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec9a5e0-b133-4598-941c-540dbcb8d5d9 None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:01.624874 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec9a5e0-b133-4598-941c-540dbcb8d5d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:01.649068 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:57:01.664774 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.auth.core [None req-0ec9a5e0-b133-4598-941c-540dbcb8d5d9 None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 11:57:01.670655 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 35/64] 10.222.0.23 () {58 vars in 1023 bytes} [Thu May 19 11:57:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 11:57:01.670780 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-0ec9a5e0-b133-4598-941c-540dbcb8d5d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 11:57:01.714142 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cdc2823-00de-4c20-b35a-bf9857fdab70 None None] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:57:01.776183 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 30/65] 10.222.0.23 () {58 vars in 1023 bytes} [Thu May 19 11:57:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 11:57:01.776183 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e81122-57cf-452d-b5a1-84605f21eb85 None None] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:01.776183 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e81122-57cf-452d-b5a1-84605f21eb85 None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:01.776183 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e81122-57cf-452d-b5a1-84605f21eb85 None None] PATH_INFO: `/` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:01.776183 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 31/66] 10.222.0.23 () {54 vars in 926 bytes} [Thu May 19 11:57:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 11:57:01.781207 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a12f449-eb23-4a73-be59-3526b253f3f9 None None] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:01.782333 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a12f449-eb23-4a73-be59-3526b253f3f9 None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:01.782939 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a12f449-eb23-4a73-be59-3526b253f3f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:01.804913 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.auth.core [None req-8a12f449-eb23-4a73-be59-3526b253f3f9 None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 11:57:01.806448 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-8a12f449-eb23-4a73-be59-3526b253f3f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 11:57:01.877361 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cdc2823-00de-4c20-b35a-bf9857fdab70 None None] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:57:01.879595 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:57:01.883400 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 32/67] 10.222.0.23 () {58 vars in 1023 bytes} [Thu May 19 11:57:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 11:57:01.892514 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a12f449-eb23-4a73-be59-3526b253f3f9 None None] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:57:01.918939 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4b0f28-7f38-4987-b5a3-5f6b52e45827 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:57:01.920178 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4b0f28-7f38-4987-b5a3-5f6b52e45827 None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:01.920666 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4b0f28-7f38-4987-b5a3-5f6b52e45827 None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:01.921163 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4b0f28-7f38-4987-b5a3-5f6b52e45827 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:01.936536 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4b0f28-7f38-4987-b5a3-5f6b52e45827 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 11:57:01.949321 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application [None req-4e4b0f28-7f38-4987-b5a3-5f6b52e45827 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 11:57:01.949321 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 11:57:01.949321 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 11:57:01.949321 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 11:57:01.949321 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 11:57:01.949321 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 11:57:01.949321 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 11:57:01.949321 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 11:57:01.949321 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 11:57:01.949321 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 11:57:01.949321 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 11:57:01.949321 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 11:57:01.949321 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 11:57:01.949321 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 11:57:01.949321 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 11:57:01.949321 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 11:57:01.951161 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 11:57:01.951161 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 11:57:01.951161 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 11:57:01.951161 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 11:57:01.951161 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 11:57:01.951161 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 11:57:01.951161 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 11:57:01.951161 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 11:57:01.951161 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 11:57:01.951161 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.get_or_create( May 19 11:57:01.951161 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 11:57:01.951161 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application with Lock( May 19 11:57:01.951161 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 11:57:01.951161 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._enter() May 19 11:57:01.951161 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 11:57:01.951161 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 11:57:01.951969 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 11:57:01.951969 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.creator() May 19 11:57:01.951969 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 11:57:01.951969 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application created_value = creator( May 19 11:57:01.951969 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 11:57:01.951969 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 11:57:01.951969 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 11:57:01.951969 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 11:57:01.951969 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 11:57:01.951969 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 11:57:01.951969 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 11:57:01.951969 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application May 19 11:57:01.952559 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 36/68] 10.222.0.23 () {56 vars in 1099 bytes} [Thu May 19 11:57:01 2022] GET /identity/v3/roles/member => generated 84 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 11:57:01.954239 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4b0f28-7f38-4987-b5a3-5f6b52e45827 None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:57:01.983104 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:57:02.026804 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a12f449-eb23-4a73-be59-3526b253f3f9 None None] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:57:02.067063 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e8dffe-02c8-498a-b830-104f0d8c5038 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:57:02.068168 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e8dffe-02c8-498a-b830-104f0d8c5038 None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:02.068616 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e8dffe-02c8-498a-b830-104f0d8c5038 None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:02.069059 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e8dffe-02c8-498a-b830-104f0d8c5038 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:02.073264 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4b0f28-7f38-4987-b5a3-5f6b52e45827 None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:57:02.081840 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e8dffe-02c8-498a-b830-104f0d8c5038 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 11:57:02.090542 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application [None req-78e8dffe-02c8-498a-b830-104f0d8c5038 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 11:57:02.090542 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 11:57:02.090542 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 11:57:02.090542 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 11:57:02.090542 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 11:57:02.090542 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 11:57:02.090542 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 11:57:02.090542 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 11:57:02.090542 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 11:57:02.090542 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 11:57:02.090542 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 11:57:02.090542 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 11:57:02.090542 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 11:57:02.090542 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 11:57:02.090542 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 11:57:02.090542 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 11:57:02.091577 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 11:57:02.091577 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 11:57:02.091577 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 11:57:02.091577 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 11:57:02.091577 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 11:57:02.091577 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 11:57:02.091577 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 11:57:02.091577 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 11:57:02.091577 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 11:57:02.091577 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.get_or_create( May 19 11:57:02.091577 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 11:57:02.091577 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application with Lock( May 19 11:57:02.091577 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 11:57:02.091577 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._enter() May 19 11:57:02.091577 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 11:57:02.091577 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 11:57:02.092373 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 11:57:02.092373 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.creator() May 19 11:57:02.092373 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 11:57:02.092373 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application created_value = creator( May 19 11:57:02.092373 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 11:57:02.092373 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 11:57:02.092373 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 11:57:02.092373 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 11:57:02.092373 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 11:57:02.092373 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 11:57:02.092373 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 11:57:02.092373 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application May 19 11:57:02.093383 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 33/69] 10.222.0.23 () {56 vars in 1096 bytes} [Thu May 19 11:57:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 11:57:02.094248 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e8dffe-02c8-498a-b830-104f0d8c5038 None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:57:02.115858 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3722f5af-ae01-46bc-943d-a3233b997a69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:57:02.116680 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3722f5af-ae01-46bc-943d-a3233b997a69 None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:02.117154 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3722f5af-ae01-46bc-943d-a3233b997a69 None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:02.117438 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3722f5af-ae01-46bc-943d-a3233b997a69 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:02.131679 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3722f5af-ae01-46bc-943d-a3233b997a69 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 11:57:02.142406 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application [None req-3722f5af-ae01-46bc-943d-a3233b997a69 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 11:57:02.142406 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 11:57:02.142406 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 11:57:02.142406 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 11:57:02.142406 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 11:57:02.142406 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 11:57:02.142406 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 11:57:02.142406 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 11:57:02.142406 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 11:57:02.142406 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 11:57:02.142406 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 11:57:02.142406 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 11:57:02.142406 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 11:57:02.142406 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 11:57:02.142406 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 11:57:02.142406 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 11:57:02.143344 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 11:57:02.143344 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 11:57:02.143344 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 11:57:02.143344 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 11:57:02.143344 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 11:57:02.143344 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 11:57:02.143344 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 11:57:02.143344 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 11:57:02.143344 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 11:57:02.143344 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.get_or_create( May 19 11:57:02.143344 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 11:57:02.143344 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application with Lock( May 19 11:57:02.143344 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 11:57:02.143344 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._enter() May 19 11:57:02.143344 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 11:57:02.143344 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 11:57:02.144100 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 11:57:02.144100 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.creator() May 19 11:57:02.144100 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 11:57:02.144100 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application created_value = creator( May 19 11:57:02.144100 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 11:57:02.144100 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 11:57:02.144100 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 11:57:02.144100 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 11:57:02.144100 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 11:57:02.144100 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 11:57:02.144100 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 11:57:02.144100 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application May 19 11:57:02.144673 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 37/70] 10.222.0.23 () {56 vars in 1114 bytes} [Thu May 19 11:57:01 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 11:57:02.145942 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcd0a8b-71e0-4758-87ea-9a0341e2b8e9 None None] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:02.146855 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcd0a8b-71e0-4758-87ea-9a0341e2b8e9 None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:02.147623 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcd0a8b-71e0-4758-87ea-9a0341e2b8e9 None None] PATH_INFO: `/` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:02.149418 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 38/71] 10.222.0.23 () {54 vars in 926 bytes} [Thu May 19 11:57:02 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 11:57:02.150785 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adcd0a8b-71e0-4758-87ea-9a0341e2b8e9 None None] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:57:02.182402 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:57:02.204594 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e8dffe-02c8-498a-b830-104f0d8c5038 None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:57:02.231954 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:57:02.243866 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646cd5d6-5351-40ed-bac7-dc2a40f6e30f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:57:02.244548 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646cd5d6-5351-40ed-bac7-dc2a40f6e30f None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:02.245345 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646cd5d6-5351-40ed-bac7-dc2a40f6e30f None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:02.245345 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646cd5d6-5351-40ed-bac7-dc2a40f6e30f None admin] PATH_INFO: `/v3/roles` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:02.246078 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646cd5d6-5351-40ed-bac7-dc2a40f6e30f None admin] RBAC: Adding query filter params (name=member) {{(pid=80539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 11:57:02.246386 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646cd5d6-5351-40ed-bac7-dc2a40f6e30f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:57:02.247823 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646cd5d6-5351-40ed-bac7-dc2a40f6e30f None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:57:02.257753 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 34/72] 10.222.0.23 () {56 vars in 1101 bytes} [Thu May 19 11:57:02 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:57:02.258461 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646cd5d6-5351-40ed-bac7-dc2a40f6e30f None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:57:02.267727 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adcd0a8b-71e0-4758-87ea-9a0341e2b8e9 None None] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:57:02.303660 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb374f6-40cd-4ec1-80c5-61c97c0e6151 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:57:02.304292 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb374f6-40cd-4ec1-80c5-61c97c0e6151 None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:02.304588 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb374f6-40cd-4ec1-80c5-61c97c0e6151 None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:02.304875 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb374f6-40cd-4ec1-80c5-61c97c0e6151 None admin] PATH_INFO: `/v3/roles` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:02.305642 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb374f6-40cd-4ec1-80c5-61c97c0e6151 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 11:57:02.305957 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb374f6-40cd-4ec1-80c5-61c97c0e6151 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:57:02.307383 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb374f6-40cd-4ec1-80c5-61c97c0e6151 None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:57:02.316741 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 39/73] 10.222.0.23 () {56 vars in 1099 bytes} [Thu May 19 11:57:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:57:02.318061 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb01907-1ed5-4692-90b9-b1d62c9f81df None None] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:02.318416 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb01907-1ed5-4692-90b9-b1d62c9f81df None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:02.318647 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb01907-1ed5-4692-90b9-b1d62c9f81df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:02.336646 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.auth.core [None req-cfb01907-1ed5-4692-90b9-b1d62c9f81df None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 11:57:02.337925 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-cfb01907-1ed5-4692-90b9-b1d62c9f81df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 11:57:02.362814 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646cd5d6-5351-40ed-bac7-dc2a40f6e30f None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:57:02.399812 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b769f5-1544-4370-8d27-5c0a5d8a9995 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:57:02.400520 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b769f5-1544-4370-8d27-5c0a5d8a9995 None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:02.400812 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b769f5-1544-4370-8d27-5c0a5d8a9995 None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:02.401097 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b769f5-1544-4370-8d27-5c0a5d8a9995 None admin] PATH_INFO: `/v3/roles` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:02.401858 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b769f5-1544-4370-8d27-5c0a5d8a9995 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 11:57:02.402179 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b769f5-1544-4370-8d27-5c0a5d8a9995 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:57:02.403593 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b769f5-1544-4370-8d27-5c0a5d8a9995 None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:57:02.410094 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 40/74] 10.222.0.23 () {58 vars in 1023 bytes} [Thu May 19 11:57:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 11:57:02.412038 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b958d2-2afb-4faf-a3e5-a1eeab5cc179 None None] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:02.412038 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b958d2-2afb-4faf-a3e5-a1eeab5cc179 None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:02.412205 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b958d2-2afb-4faf-a3e5-a1eeab5cc179 None None] PATH_INFO: `/` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:02.413420 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 41/75] 10.222.0.23 () {54 vars in 926 bytes} [Thu May 19 11:57:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 11:57:02.414213 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b958d2-2afb-4faf-a3e5-a1eeab5cc179 None None] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:57:02.418415 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 35/76] 10.222.0.23 () {56 vars in 1111 bytes} [Thu May 19 11:57:02 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:57:02.419125 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b769f5-1544-4370-8d27-5c0a5d8a9995 None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:57:02.441770 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:57:02.476041 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:57:02.532535 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b769f5-1544-4370-8d27-5c0a5d8a9995 None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:57:02.538466 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b958d2-2afb-4faf-a3e5-a1eeab5cc179 None None] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:57:02.568261 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b747c4-6c22-4277-8b4c-5475dbdb79af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:57:02.569139 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b747c4-6c22-4277-8b4c-5475dbdb79af None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:02.569428 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b747c4-6c22-4277-8b4c-5475dbdb79af None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:02.569709 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b747c4-6c22-4277-8b4c-5475dbdb79af None admin] PATH_INFO: `/v3/users/11855e565a88475189d279de9469c271` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:02.571165 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b747c4-6c22-4277-8b4c-5475dbdb79af None admin] RBAC: Authorizing `identity:get_user(user_id=11855e565a88475189d279de9469c271)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:57:02.572645 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b747c4-6c22-4277-8b4c-5475dbdb79af None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:57:02.574474 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 36/77] 10.222.0.23 () {56 vars in 1177 bytes} [Thu May 19 11:57:02 2022] GET /identity/v3/users/11855e565a88475189d279de9469c271 => generated 247 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:57:02.575408 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401a5eae-6c98-4c31-aaf9-e7ec527739c6 None None] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:02.575700 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401a5eae-6c98-4c31-aaf9-e7ec527739c6 None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:02.576006 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401a5eae-6c98-4c31-aaf9-e7ec527739c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:02.580648 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bf492a-1fbf-48d4-8010-6371b92310e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:57:02.581530 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bf492a-1fbf-48d4-8010-6371b92310e6 None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:02.581811 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bf492a-1fbf-48d4-8010-6371b92310e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:02.582084 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bf492a-1fbf-48d4-8010-6371b92310e6 None admin] PATH_INFO: `/v3/users/267c35a521bb473ea2784646137bebda` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:02.594465 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.auth.core [None req-401a5eae-6c98-4c31-aaf9-e7ec527739c6 None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 11:57:02.595860 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-401a5eae-6c98-4c31-aaf9-e7ec527739c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 11:57:02.597068 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28bf492a-1fbf-48d4-8010-6371b92310e6 None admin] RBAC: Authorizing `identity:get_user(user_id=267c35a521bb473ea2784646137bebda)` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:57:02.599767 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28bf492a-1fbf-48d4-8010-6371b92310e6 None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:57:02.602513 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 42/78] 10.222.0.23 () {56 vars in 1177 bytes} [Thu May 19 11:57:02 2022] GET /identity/v3/users/267c35a521bb473ea2784646137bebda => generated 275 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:57:02.603301 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bf492a-1fbf-48d4-8010-6371b92310e6 None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:57:02.668789 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 37/79] 10.222.0.23 () {58 vars in 1023 bytes} [Thu May 19 11:57:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 11:57:02.669665 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401a5eae-6c98-4c31-aaf9-e7ec527739c6 None None] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:57:02.682559 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:57:02.714935 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bf492a-1fbf-48d4-8010-6371b92310e6 None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:57:02.721984 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:57:02.751263 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015cac27-8116-4eda-aacf-4d5bd510c700 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:57:02.752186 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015cac27-8116-4eda-aacf-4d5bd510c700 None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:02.752478 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015cac27-8116-4eda-aacf-4d5bd510c700 None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:02.752762 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015cac27-8116-4eda-aacf-4d5bd510c700 None admin] PATH_INFO: `/v3/users/267c35a521bb473ea2784646137bebda` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:02.754007 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015cac27-8116-4eda-aacf-4d5bd510c700 None admin] RBAC: Authorizing `identity:get_user(user_id=267c35a521bb473ea2784646137bebda)` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:57:02.755481 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015cac27-8116-4eda-aacf-4d5bd510c700 None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:57:02.757162 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 43/80] 10.222.0.23 () {56 vars in 1177 bytes} [Thu May 19 11:57:02 2022] GET /identity/v3/users/267c35a521bb473ea2784646137bebda => generated 275 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:57:02.757951 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015cac27-8116-4eda-aacf-4d5bd510c700 None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:57:02.778616 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401a5eae-6c98-4c31-aaf9-e7ec527739c6 None None] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:57:02.813503 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4156a45c-fa2e-4641-a21a-83b27c2df4e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:57:02.814312 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4156a45c-fa2e-4641-a21a-83b27c2df4e1 None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:02.814608 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4156a45c-fa2e-4641-a21a-83b27c2df4e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:02.814889 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4156a45c-fa2e-4641-a21a-83b27c2df4e1 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:02.825729 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4156a45c-fa2e-4641-a21a-83b27c2df4e1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 11:57:02.834748 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application [None req-4156a45c-fa2e-4641-a21a-83b27c2df4e1 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 11:57:02.834748 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 11:57:02.834748 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 11:57:02.834748 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 11:57:02.834748 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 11:57:02.834748 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 11:57:02.834748 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 11:57:02.834748 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 11:57:02.834748 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 11:57:02.834748 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 11:57:02.834748 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 11:57:02.834748 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 11:57:02.834748 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 11:57:02.834748 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 11:57:02.834748 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 11:57:02.834748 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 11:57:02.835758 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 11:57:02.835758 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 11:57:02.835758 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 11:57:02.835758 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 11:57:02.835758 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 11:57:02.835758 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 11:57:02.835758 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 11:57:02.835758 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 11:57:02.835758 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 11:57:02.835758 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.get_or_create( May 19 11:57:02.835758 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 11:57:02.835758 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application with Lock( May 19 11:57:02.835758 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 11:57:02.835758 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._enter() May 19 11:57:02.835758 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 11:57:02.835758 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 11:57:02.836675 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 11:57:02.836675 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.creator() May 19 11:57:02.836675 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 11:57:02.836675 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application created_value = creator( May 19 11:57:02.836675 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 11:57:02.836675 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 11:57:02.836675 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 11:57:02.836675 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 11:57:02.836675 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 11:57:02.836675 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 11:57:02.836675 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 11:57:02.836675 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application May 19 11:57:02.836675 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 38/81] 10.222.0.23 () {56 vars in 1099 bytes} [Thu May 19 11:57:02 2022] GET /identity/v3/roles/member => generated 84 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 11:57:02.837369 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4156a45c-fa2e-4641-a21a-83b27c2df4e1 None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:57:02.866627 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015cac27-8116-4eda-aacf-4d5bd510c700 None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:57:02.900787 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151e3019-c12b-425e-a82b-dac230e2ddaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:57:02.901445 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151e3019-c12b-425e-a82b-dac230e2ddaa None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:02.901733 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151e3019-c12b-425e-a82b-dac230e2ddaa None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:02.902014 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151e3019-c12b-425e-a82b-dac230e2ddaa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:02.912997 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:57:02.944585 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4156a45c-fa2e-4641-a21a-83b27c2df4e1 None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:57:02.955082 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:57:02.980620 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa240d70-c40b-4b36-8fc0-f0a00a1cfd86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:57:02.981283 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa240d70-c40b-4b36-8fc0-f0a00a1cfd86 None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:02.981627 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa240d70-c40b-4b36-8fc0-f0a00a1cfd86 None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:02.981924 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa240d70-c40b-4b36-8fc0-f0a00a1cfd86 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:02.982235 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151e3019-c12b-425e-a82b-dac230e2ddaa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:57:02.983750 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151e3019-c12b-425e-a82b-dac230e2ddaa None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:57:03.051692 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa240d70-c40b-4b36-8fc0-f0a00a1cfd86 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:57:03.053245 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa240d70-c40b-4b36-8fc0-f0a00a1cfd86 None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:57:03.092518 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 44/82] 10.222.0.23 () {58 vars in 1282 bytes} [Thu May 19 11:57:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 11:57:03.093293 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151e3019-c12b-425e-a82b-dac230e2ddaa None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:57:03.143860 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:57:03.153601 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 39/83] 10.222.0.23 () {58 vars in 1282 bytes} [Thu May 19 11:57:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 11:57:03.153601 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa240d70-c40b-4b36-8fc0-f0a00a1cfd86 None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:57:03.165821 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:57:03.204731 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151e3019-c12b-425e-a82b-dac230e2ddaa None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:57:03.239859 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6c0a13-97cb-440a-99c5-9b6d7243b57c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:57:03.240680 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6c0a13-97cb-440a-99c5-9b6d7243b57c None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:03.240973 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6c0a13-97cb-440a-99c5-9b6d7243b57c None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:03.241262 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6c0a13-97cb-440a-99c5-9b6d7243b57c None admin] PATH_INFO: `/v3/domains/default` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:03.242535 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6c0a13-97cb-440a-99c5-9b6d7243b57c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:57:03.244054 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6c0a13-97cb-440a-99c5-9b6d7243b57c None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:57:03.245690 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 45/84] 10.222.0.23 () {56 vars in 1108 bytes} [Thu May 19 11:57:03 2022] GET /identity/v3/domains/default => generated 199 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:57:03.246469 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6c0a13-97cb-440a-99c5-9b6d7243b57c None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:57:03.258605 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa240d70-c40b-4b36-8fc0-f0a00a1cfd86 None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:57:03.293512 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a66ae89-5c87-46cb-aa62-059087e6e2ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:57:03.294170 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a66ae89-5c87-46cb-aa62-059087e6e2ea None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:03.294466 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a66ae89-5c87-46cb-aa62-059087e6e2ea None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:03.294749 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a66ae89-5c87-46cb-aa62-059087e6e2ea None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:03.355861 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6c0a13-97cb-440a-99c5-9b6d7243b57c None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:57:03.366818 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:57:03.369055 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a66ae89-5c87-46cb-aa62-059087e6e2ea None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:57:03.370552 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a66ae89-5c87-46cb-aa62-059087e6e2ea None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:57:03.381727 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:57:03.392196 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d57de8-f6c9-4612-ab18-ab6f1f4ad186 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:57:03.392881 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d57de8-f6c9-4612-ab18-ab6f1f4ad186 None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:03.393171 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d57de8-f6c9-4612-ab18-ab6f1f4ad186 None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:03.393499 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d57de8-f6c9-4612-ab18-ab6f1f4ad186 None admin] PATH_INFO: `/v3/roles` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:03.394264 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d57de8-f6c9-4612-ab18-ab6f1f4ad186 None admin] RBAC: Adding query filter params (name=member) {{(pid=80540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 11:57:03.394581 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d57de8-f6c9-4612-ab18-ab6f1f4ad186 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:57:03.396052 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d57de8-f6c9-4612-ab18-ab6f1f4ad186 None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:57:03.408509 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 46/85] 10.222.0.23 () {56 vars in 1101 bytes} [Thu May 19 11:57:03 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:57:03.410512 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d57de8-f6c9-4612-ab18-ab6f1f4ad186 None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:57:03.475308 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 40/86] 10.222.0.23 () {58 vars in 1282 bytes} [Thu May 19 11:57:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 11:57:03.476207 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a66ae89-5c87-46cb-aa62-059087e6e2ea None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:57:03.518184 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d57de8-f6c9-4612-ab18-ab6f1f4ad186 None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:57:03.555126 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8949c561-a6b8-4b3c-8fd7-1fc445a45bfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:57:03.556043 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8949c561-a6b8-4b3c-8fd7-1fc445a45bfb None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:03.556333 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8949c561-a6b8-4b3c-8fd7-1fc445a45bfb None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:03.556609 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8949c561-a6b8-4b3c-8fd7-1fc445a45bfb None admin] PATH_INFO: `/v3/projects/64092131887b4859ae47a1f4c3e0d24a` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:03.557801 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8949c561-a6b8-4b3c-8fd7-1fc445a45bfb None admin] RBAC: Authorizing `identity:get_project(project_id=64092131887b4859ae47a1f4c3e0d24a)` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:57:03.559274 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8949c561-a6b8-4b3c-8fd7-1fc445a45bfb None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:57:03.561012 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 47/87] 10.222.0.23 () {56 vars in 1186 bytes} [Thu May 19 11:57:03 2022] GET /identity/v3/projects/64092131887b4859ae47a1f4c3e0d24a => generated 298 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:57:03.564667 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8949c561-a6b8-4b3c-8fd7-1fc445a45bfb None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:57:03.585380 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a66ae89-5c87-46cb-aa62-059087e6e2ea None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:57:03.594742 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:57:03.609570 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:57:03.625159 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841c85c2-1a4b-4c5a-9343-69e43f0bca14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:57:03.627399 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841c85c2-1a4b-4c5a-9343-69e43f0bca14 None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:03.627708 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841c85c2-1a4b-4c5a-9343-69e43f0bca14 None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:03.628022 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841c85c2-1a4b-4c5a-9343-69e43f0bca14 None admin] PATH_INFO: `/v3/projects/64092131887b4859ae47a1f4c3e0d24a` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:03.629277 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841c85c2-1a4b-4c5a-9343-69e43f0bca14 None admin] RBAC: Authorizing `identity:get_project(project_id=64092131887b4859ae47a1f4c3e0d24a)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:57:03.630754 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841c85c2-1a4b-4c5a-9343-69e43f0bca14 None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:57:03.632511 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 41/88] 10.222.0.23 () {56 vars in 1186 bytes} [Thu May 19 11:57:03 2022] GET /identity/v3/projects/64092131887b4859ae47a1f4c3e0d24a => generated 298 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:57:03.633287 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841c85c2-1a4b-4c5a-9343-69e43f0bca14 None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:57:03.669671 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8949c561-a6b8-4b3c-8fd7-1fc445a45bfb None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:57:03.706569 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb35179f-35a4-4016-8b88-2f0344d22930 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:57:03.707247 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb35179f-35a4-4016-8b88-2f0344d22930 None admin] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:03.707537 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb35179f-35a4-4016-8b88-2f0344d22930 None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:03.707841 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb35179f-35a4-4016-8b88-2f0344d22930 None admin] PATH_INFO: `/v3/users` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:03.708561 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb35179f-35a4-4016-8b88-2f0344d22930 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:57:03.710034 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb35179f-35a4-4016-8b88-2f0344d22930 None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:57:03.734489 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841c85c2-1a4b-4c5a-9343-69e43f0bca14 None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:57:03.770565 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59ef54f-96a2-4153-a172-6e790870e520 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:57:03.771438 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59ef54f-96a2-4153-a172-6e790870e520 None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:03.771864 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59ef54f-96a2-4153-a172-6e790870e520 None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:03.772144 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59ef54f-96a2-4153-a172-6e790870e520 None admin] PATH_INFO: `/v3/users/267c35a521bb473ea2784646137bebda` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:03.773509 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59ef54f-96a2-4153-a172-6e790870e520 None admin] RBAC: Authorizing `identity:get_user(user_id=267c35a521bb473ea2784646137bebda)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:57:03.774957 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59ef54f-96a2-4153-a172-6e790870e520 None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:57:03.776672 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 42/89] 10.222.0.23 () {56 vars in 1177 bytes} [Thu May 19 11:57:03 2022] GET /identity/v3/users/267c35a521bb473ea2784646137bebda => generated 275 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:57:03.777452 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59ef54f-96a2-4153-a172-6e790870e520 None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:57:03.787649 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 48/90] 10.222.0.23 () {60 vars in 1132 bytes} [Thu May 19 11:57:03 2022] POST /identity/v3/users => generated 289 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 11:57:03.799087 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb35179f-35a4-4016-8b88-2f0344d22930 None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:57:03.831064 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:57:03.894002 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59ef54f-96a2-4153-a172-6e790870e520 None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:57:03.940774 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:57:03.945026 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc2c41a-ffae-4a67-ae82-a12ff900cf02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:57:03.945952 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc2c41a-ffae-4a67-ae82-a12ff900cf02 None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:03.946240 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc2c41a-ffae-4a67-ae82-a12ff900cf02 None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:03.946523 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc2c41a-ffae-4a67-ae82-a12ff900cf02 None admin] PATH_INFO: `/v3/projects/64092131887b4859ae47a1f4c3e0d24a` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:03.948020 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc2c41a-ffae-4a67-ae82-a12ff900cf02 None admin] RBAC: Authorizing `identity:get_project(project_id=64092131887b4859ae47a1f4c3e0d24a)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:57:03.955973 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc2c41a-ffae-4a67-ae82-a12ff900cf02 None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:57:03.956861 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb35179f-35a4-4016-8b88-2f0344d22930 None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:57:03.956964 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 43/91] 10.222.0.23 () {56 vars in 1186 bytes} [Thu May 19 11:57:03 2022] GET /identity/v3/projects/64092131887b4859ae47a1f4c3e0d24a => generated 298 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:57:03.956964 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc2c41a-ffae-4a67-ae82-a12ff900cf02 None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:57:03.993959 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bffe5b5-96e8-4040-94a9-251223aa1c2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:57:03.994625 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bffe5b5-96e8-4040-94a9-251223aa1c2f None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:03.994912 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bffe5b5-96e8-4040-94a9-251223aa1c2f None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:03.995196 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bffe5b5-96e8-4040-94a9-251223aa1c2f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:03.996077 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bffe5b5-96e8-4040-94a9-251223aa1c2f None admin] RBAC: Adding query filter params (role.id=9db5671c65d3414ea2682afa2ccf1403, scope.project.id=64092131887b4859ae47a1f4c3e0d24a, user.id=11855e565a88475189d279de9469c271) {{(pid=80540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 11:57:03.996392 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bffe5b5-96e8-4040-94a9-251223aa1c2f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:57:03.999961 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bffe5b5-96e8-4040-94a9-251223aa1c2f None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:57:04.006742 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 49/92] 10.222.0.23 () {56 vars in 1374 bytes} [Thu May 19 11:57:03 2022] GET /identity/v3/role_assignments?user.id=11855e565a88475189d279de9469c271&scope.project.id=64092131887b4859ae47a1f4c3e0d24a&role.id=9db5671c65d3414ea2682afa2ccf1403 => generated 259 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:57:04.007315 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bffe5b5-96e8-4040-94a9-251223aa1c2f None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:57:04.131078 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc2c41a-ffae-4a67-ae82-a12ff900cf02 None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:57:04.150021 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:57:04.169490 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bffe5b5-96e8-4040-94a9-251223aa1c2f None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:57:04.179200 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83fc247f-da9c-4e21-a53e-86307f478caf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:57:04.180201 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fc247f-da9c-4e21-a53e-86307f478caf None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:04.180659 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fc247f-da9c-4e21-a53e-86307f478caf None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:04.181104 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fc247f-da9c-4e21-a53e-86307f478caf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:04.182144 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83fc247f-da9c-4e21-a53e-86307f478caf None admin] RBAC: Adding query filter params (role.id=259eb400cdb4439db72d14456ae9cd08, scope.project.id=64092131887b4859ae47a1f4c3e0d24a, user.id=267c35a521bb473ea2784646137bebda) {{(pid=80539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 11:57:04.182618 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83fc247f-da9c-4e21-a53e-86307f478caf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:57:04.184397 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83fc247f-da9c-4e21-a53e-86307f478caf None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:57:04.197943 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 44/93] 10.222.0.23 () {56 vars in 1374 bytes} [Thu May 19 11:57:03 2022] GET /identity/v3/role_assignments?user.id=267c35a521bb473ea2784646137bebda&scope.project.id=64092131887b4859ae47a1f4c3e0d24a&role.id=259eb400cdb4439db72d14456ae9cd08 => generated 259 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:57:04.198830 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83fc247f-da9c-4e21-a53e-86307f478caf None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:57:04.235270 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9158e6e-d242-4237-a161-a38164b97fa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:57:04.236175 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9158e6e-d242-4237-a161-a38164b97fa3 None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:04.236574 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9158e6e-d242-4237-a161-a38164b97fa3 None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:04.236988 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9158e6e-d242-4237-a161-a38164b97fa3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:04.271500 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:57:04.320731 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9158e6e-d242-4237-a161-a38164b97fa3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:57:04.322427 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9158e6e-d242-4237-a161-a38164b97fa3 None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:57:04.343658 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83fc247f-da9c-4e21-a53e-86307f478caf None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:57:04.396406 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4db23a-12ec-445d-a202-e9ba50e17cb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:57:04.397469 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4db23a-12ec-445d-a202-e9ba50e17cb4 None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:04.397913 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4db23a-12ec-445d-a202-e9ba50e17cb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:04.398340 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4db23a-12ec-445d-a202-e9ba50e17cb4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:04.399384 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4db23a-12ec-445d-a202-e9ba50e17cb4 None admin] RBAC: Adding query filter params (role.id=8215583bb4224c75b093ae8e3993eb9f, scope.project.id=64092131887b4859ae47a1f4c3e0d24a, user.id=267c35a521bb473ea2784646137bebda) {{(pid=80539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 11:57:04.399892 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4db23a-12ec-445d-a202-e9ba50e17cb4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:57:04.401958 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4db23a-12ec-445d-a202-e9ba50e17cb4 None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:57:04.414182 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 45/94] 10.222.0.23 () {56 vars in 1374 bytes} [Thu May 19 11:57:04 2022] GET /identity/v3/role_assignments?user.id=267c35a521bb473ea2784646137bebda&scope.project.id=64092131887b4859ae47a1f4c3e0d24a&role.id=8215583bb4224c75b093ae8e3993eb9f => generated 259 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:57:04.439476 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 50/95] 10.222.0.23 () {58 vars in 1282 bytes} [Thu May 19 11:57:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 432 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 11:57:04.445019 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4db23a-12ec-445d-a202-e9ba50e17cb4 None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:57:04.495168 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:57:04.577110 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4db23a-12ec-445d-a202-e9ba50e17cb4 None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:57:04.628515 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f3c231-f3de-4385-afda-45119e2ba7a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:57:04.629436 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f3c231-f3de-4385-afda-45119e2ba7a0 None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:04.629730 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f3c231-f3de-4385-afda-45119e2ba7a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:04.630019 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f3c231-f3de-4385-afda-45119e2ba7a0 None admin] PATH_INFO: `/v3/projects/4aeddbfe954d472aa273e1df1efca1fd` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:04.633230 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f3c231-f3de-4385-afda-45119e2ba7a0 None admin] RBAC: Authorizing `identity:get_project(project_id=4aeddbfe954d472aa273e1df1efca1fd)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:57:04.634765 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f3c231-f3de-4385-afda-45119e2ba7a0 None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:57:04.636654 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 46/96] 10.222.0.23 () {56 vars in 1186 bytes} [Thu May 19 11:57:04 2022] GET /identity/v3/projects/4aeddbfe954d472aa273e1df1efca1fd => generated 312 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:57:04.643581 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9158e6e-d242-4237-a161-a38164b97fa3 None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:57:04.745918 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:57:04.801600 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9158e6e-d242-4237-a161-a38164b97fa3 None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:57:04.856592 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f94d918-b45a-4029-8aef-70669b711869 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:57:04.857280 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f94d918-b45a-4029-8aef-70669b711869 None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:04.857695 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f94d918-b45a-4029-8aef-70669b711869 None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:04.857980 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f94d918-b45a-4029-8aef-70669b711869 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:04.858832 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f94d918-b45a-4029-8aef-70669b711869 None admin] RBAC: Adding query filter params (role.id=259eb400cdb4439db72d14456ae9cd08, scope.project.id=4aeddbfe954d472aa273e1df1efca1fd, user.id=267c35a521bb473ea2784646137bebda) {{(pid=80540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 11:57:04.859154 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f94d918-b45a-4029-8aef-70669b711869 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:57:04.861642 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f94d918-b45a-4029-8aef-70669b711869 None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:57:04.868077 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 51/97] 10.222.0.23 () {56 vars in 1374 bytes} [Thu May 19 11:57:04 2022] GET /identity/v3/role_assignments?user.id=267c35a521bb473ea2784646137bebda&scope.project.id=4aeddbfe954d472aa273e1df1efca1fd&role.id=259eb400cdb4439db72d14456ae9cd08 => generated 259 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:57:06.967593 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83de3f9-45e1-4133-8b47-2a52de96cc47 None None] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:06.968153 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83de3f9-45e1-4133-8b47-2a52de96cc47 None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:06.968227 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83de3f9-45e1-4133-8b47-2a52de96cc47 None None] PATH_INFO: `/` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:06.969904 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 47/98] 10.222.0.23 () {54 vars in 926 bytes} [Thu May 19 11:57:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 11:57:06.975607 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f387d9-d56e-479b-8bc3-48b230d298a5 None None] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:06.976534 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f387d9-d56e-479b-8bc3-48b230d298a5 None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:06.977016 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f387d9-d56e-479b-8bc3-48b230d298a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:07.004915 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.auth.core [None req-40f387d9-d56e-479b-8bc3-48b230d298a5 None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 11:57:07.006499 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-40f387d9-d56e-479b-8bc3-48b230d298a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 11:57:07.093660 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 52/99] 10.222.0.23 () {58 vars in 1023 bytes} [Thu May 19 11:57:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 11:57:07.133019 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83de3f9-45e1-4133-8b47-2a52de96cc47 None None] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:57:07.167318 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e03bdc-dc3e-4aa6-b902-d236bfbcd3a4 None None] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:07.167993 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e03bdc-dc3e-4aa6-b902-d236bfbcd3a4 None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:07.168481 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e03bdc-dc3e-4aa6-b902-d236bfbcd3a4 None None] PATH_INFO: `/` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:07.170381 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 53/100] 10.222.0.23 () {54 vars in 926 bytes} [Thu May 19 11:57:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 11:57:07.184634 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1af389-e98a-49fc-af3b-206bb996a4fe None None] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:07.185284 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1af389-e98a-49fc-af3b-206bb996a4fe None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:07.185856 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1af389-e98a-49fc-af3b-206bb996a4fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:07.209231 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.auth.core [None req-3c1af389-e98a-49fc-af3b-206bb996a4fe None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 11:57:07.211004 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-3c1af389-e98a-49fc-af3b-206bb996a4fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 11:57:07.306963 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:57:07.329322 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83de3f9-45e1-4133-8b47-2a52de96cc47 None None] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:57:07.343500 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 54/101] 10.222.0.23 () {58 vars in 1023 bytes} [Thu May 19 11:57:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 11:57:07.354408 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:57:07.379951 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1af389-e98a-49fc-af3b-206bb996a4fe None None] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:57:07.464524 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364961f9-5354-49bb-8409-ad6d0e505f51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:57:07.465264 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364961f9-5354-49bb-8409-ad6d0e505f51 None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:07.465551 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364961f9-5354-49bb-8409-ad6d0e505f51 None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:07.465828 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364961f9-5354-49bb-8409-ad6d0e505f51 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:07.516844 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364961f9-5354-49bb-8409-ad6d0e505f51 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 11:57:07.524994 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application [None req-364961f9-5354-49bb-8409-ad6d0e505f51 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 19 11:57:07.524994 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 11:57:07.524994 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 11:57:07.524994 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 11:57:07.524994 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 11:57:07.524994 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 11:57:07.524994 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 11:57:07.524994 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 11:57:07.524994 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 11:57:07.524994 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 11:57:07.524994 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 11:57:07.524994 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 11:57:07.524994 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 11:57:07.524994 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 11:57:07.524994 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 11:57:07.524994 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 11:57:07.526055 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 11:57:07.526055 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 11:57:07.526055 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 11:57:07.526055 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 11:57:07.526055 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 11:57:07.526055 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 11:57:07.526055 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 11:57:07.526055 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 11:57:07.526055 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 11:57:07.526055 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.get_or_create( May 19 11:57:07.526055 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 11:57:07.526055 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application with Lock( May 19 11:57:07.526055 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 11:57:07.526055 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._enter() May 19 11:57:07.526055 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 11:57:07.526055 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 11:57:07.526973 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 11:57:07.526973 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.creator() May 19 11:57:07.526973 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 11:57:07.526973 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application created_value = creator( May 19 11:57:07.526973 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 11:57:07.526973 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 11:57:07.526973 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 11:57:07.526973 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 11:57:07.526973 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 11:57:07.526973 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 11:57:07.526973 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 19 11:57:07.526973 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application May 19 11:57:07.537207 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 48/102] 10.222.0.23 () {56 vars in 1099 bytes} [Thu May 19 11:57:07 2022] GET /identity/v3/roles/reader => generated 84 bytes in 405 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 11:57:07.548966 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1af389-e98a-49fc-af3b-206bb996a4fe None None] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:57:07.560885 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364961f9-5354-49bb-8409-ad6d0e505f51 None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:57:07.610366 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c0c4d9-9864-4156-9e9f-f2c816611164 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:57:07.611209 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c0c4d9-9864-4156-9e9f-f2c816611164 None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:07.611542 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c0c4d9-9864-4156-9e9f-f2c816611164 None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:07.612014 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c0c4d9-9864-4156-9e9f-f2c816611164 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:07.613464 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c0c4d9-9864-4156-9e9f-f2c816611164 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:57:07.615351 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c0c4d9-9864-4156-9e9f-f2c816611164 None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:57:07.617186 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 55/103] 10.222.0.23 () {56 vars in 1108 bytes} [Thu May 19 11:57:07 2022] GET /identity/v3/domains/default => generated 199 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 11:57:07.624606 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c0c4d9-9864-4156-9e9f-f2c816611164 None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:57:07.751992 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:57:07.855500 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364961f9-5354-49bb-8409-ad6d0e505f51 None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:57:07.859411 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c0c4d9-9864-4156-9e9f-f2c816611164 None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:57:07.893151 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585b6626-7518-442f-af50-62d27b96ec7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:57:07.893838 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585b6626-7518-442f-af50-62d27b96ec7e None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:07.894124 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585b6626-7518-442f-af50-62d27b96ec7e None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:07.894403 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585b6626-7518-442f-af50-62d27b96ec7e None admin] PATH_INFO: `/v3/roles` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:07.895164 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585b6626-7518-442f-af50-62d27b96ec7e None admin] RBAC: Adding query filter params (name=reader) {{(pid=80539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 11:57:07.895472 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585b6626-7518-442f-af50-62d27b96ec7e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:57:07.896924 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585b6626-7518-442f-af50-62d27b96ec7e None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:57:07.905987 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 49/104] 10.222.0.23 () {56 vars in 1101 bytes} [Thu May 19 11:57:07 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 346 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:57:07.907389 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6441ca37-2004-4e42-80c5-4d8a640e6aa1 None None] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:07.907685 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6441ca37-2004-4e42-80c5-4d8a640e6aa1 None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:07.907992 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6441ca37-2004-4e42-80c5-4d8a640e6aa1 None None] PATH_INFO: `/` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:07.909233 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 50/105] 10.222.0.23 () {54 vars in 926 bytes} [Thu May 19 11:57:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 11:57:07.910098 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3103d154-b9df-4379-b95b-67e7f751707e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:57:07.910290 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c623195-3c57-48ef-8346-2afbb6d647e7 None None] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:07.910682 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c623195-3c57-48ef-8346-2afbb6d647e7 None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:07.910965 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c623195-3c57-48ef-8346-2afbb6d647e7 None None] PATH_INFO: `/` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:07.911555 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3103d154-b9df-4379-b95b-67e7f751707e None admin] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:07.912076 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3103d154-b9df-4379-b95b-67e7f751707e None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:07.916133 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 51/106] 10.222.0.23 () {54 vars in 926 bytes} [Thu May 19 11:57:07 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 11:57:07.916133 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3dc1a09-428d-4a7c-b6bc-070ebb58db35 None None] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:07.916133 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3dc1a09-428d-4a7c-b6bc-070ebb58db35 None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:07.916133 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3dc1a09-428d-4a7c-b6bc-070ebb58db35 None None] PATH_INFO: `/` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:07.916133 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 52/107] 10.222.0.23 () {54 vars in 926 bytes} [Thu May 19 11:57:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 11:57:07.916493 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3103d154-b9df-4379-b95b-67e7f751707e None admin] PATH_INFO: `/v3/projects` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:07.917379 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3103d154-b9df-4379-b95b-67e7f751707e None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:57:07.918723 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3103d154-b9df-4379-b95b-67e7f751707e None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:57:07.920723 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1690fc7-a7eb-4af7-8137-05fb2a1c3e16 None None] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:07.920958 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1690fc7-a7eb-4af7-8137-05fb2a1c3e16 None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:07.921237 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1690fc7-a7eb-4af7-8137-05fb2a1c3e16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:07.941013 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.auth.core [None req-c1690fc7-a7eb-4af7-8137-05fb2a1c3e16 None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 11:57:07.942320 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-c1690fc7-a7eb-4af7-8137-05fb2a1c3e16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 11:57:07.968371 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:57:08.011941 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 56/108] 10.222.0.23 () {60 vars in 1141 bytes} [Thu May 19 11:57:07 2022] POST /identity/v3/projects => generated 302 bytes in 379 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 11:57:08.011941 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3103d154-b9df-4379-b95b-67e7f751707e None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:57:08.026828 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 53/109] 10.222.0.23 () {58 vars in 1023 bytes} [Thu May 19 11:57:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 11:57:08.028139 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdacadf-312c-4a94-bbbc-4e5c82609153 None None] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:08.029380 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdacadf-312c-4a94-bbbc-4e5c82609153 None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:08.029380 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdacadf-312c-4a94-bbbc-4e5c82609153 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:08.058766 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.auth.core [None req-ecdacadf-312c-4a94-bbbc-4e5c82609153 None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 11:57:08.060313 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-ecdacadf-312c-4a94-bbbc-4e5c82609153 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 11:57:08.141913 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 54/110] 10.222.0.23 () {58 vars in 1023 bytes} [Thu May 19 11:57:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 11:57:08.143070 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da5d3fe-dd7d-47c0-a92a-ca22ef60f03a None None] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:08.143437 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da5d3fe-dd7d-47c0-a92a-ca22ef60f03a None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:08.143812 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da5d3fe-dd7d-47c0-a92a-ca22ef60f03a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:08.158956 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3103d154-b9df-4379-b95b-67e7f751707e None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:57:08.161642 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:57:08.161874 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.auth.core [None req-2da5d3fe-dd7d-47c0-a92a-ca22ef60f03a None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 11:57:08.163212 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-2da5d3fe-dd7d-47c0-a92a-ca22ef60f03a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 11:57:08.220895 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396fa7a4-e53d-40e4-9695-f84ec2877aae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:57:08.222647 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396fa7a4-e53d-40e4-9695-f84ec2877aae None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:08.223188 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396fa7a4-e53d-40e4-9695-f84ec2877aae None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:08.223698 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396fa7a4-e53d-40e4-9695-f84ec2877aae None admin] PATH_INFO: `/v3/users/a92936fa37984cfda4b8a7a5cc363e6e` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:08.245182 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396fa7a4-e53d-40e4-9695-f84ec2877aae None admin] RBAC: Authorizing `identity:get_user(user_id=a92936fa37984cfda4b8a7a5cc363e6e)` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:57:08.247680 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396fa7a4-e53d-40e4-9695-f84ec2877aae None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:57:08.251044 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 57/111] 10.222.0.23 () {56 vars in 1177 bytes} [Thu May 19 11:57:08 2022] GET /identity/v3/users/a92936fa37984cfda4b8a7a5cc363e6e => generated 289 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:57:08.251731 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396fa7a4-e53d-40e4-9695-f84ec2877aae None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:57:08.254654 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 55/112] 10.222.0.23 () {58 vars in 1023 bytes} [Thu May 19 11:57:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 11:57:08.256138 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da5d3fe-dd7d-47c0-a92a-ca22ef60f03a None None] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:57:08.296602 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:57:08.401754 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da5d3fe-dd7d-47c0-a92a-ca22ef60f03a None None] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:57:08.405139 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396fa7a4-e53d-40e4-9695-f84ec2877aae None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:57:08.440376 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f4a902-7e18-405a-81d1-c3514f6226ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:57:08.441398 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783dd9c9-eb0e-4ca8-9852-f161ed0dddd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:57:08.441731 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f4a902-7e18-405a-81d1-c3514f6226ea None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:08.442206 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783dd9c9-eb0e-4ca8-9852-f161ed0dddd4 None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:08.442376 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f4a902-7e18-405a-81d1-c3514f6226ea None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:08.442483 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783dd9c9-eb0e-4ca8-9852-f161ed0dddd4 None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:08.442770 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783dd9c9-eb0e-4ca8-9852-f161ed0dddd4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:08.443096 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f4a902-7e18-405a-81d1-c3514f6226ea None admin] PATH_INFO: `/v3/roles/member` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:08.454558 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783dd9c9-eb0e-4ca8-9852-f161ed0dddd4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 11:57:08.455306 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f4a902-7e18-405a-81d1-c3514f6226ea None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 11:57:08.462436 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application [None req-783dd9c9-eb0e-4ca8-9852-f161ed0dddd4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 11:57:08.462436 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 11:57:08.462436 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 11:57:08.462436 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 11:57:08.462436 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 11:57:08.462436 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 11:57:08.462436 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 11:57:08.462436 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 11:57:08.462436 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 11:57:08.462436 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 11:57:08.462436 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 11:57:08.462436 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 11:57:08.462436 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 11:57:08.462436 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 11:57:08.462436 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 11:57:08.462436 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 11:57:08.463292 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 11:57:08.463292 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 11:57:08.463292 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 11:57:08.463292 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 11:57:08.463292 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 11:57:08.463292 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 11:57:08.463292 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 11:57:08.463292 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 11:57:08.463292 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 11:57:08.463292 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.get_or_create( May 19 11:57:08.463292 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 11:57:08.463292 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application with Lock( May 19 11:57:08.463292 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 11:57:08.463292 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._enter() May 19 11:57:08.463292 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 11:57:08.463292 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 11:57:08.464458 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 11:57:08.464458 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.creator() May 19 11:57:08.464458 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 11:57:08.464458 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application created_value = creator( May 19 11:57:08.464458 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 11:57:08.464458 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 11:57:08.464458 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 11:57:08.464458 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 11:57:08.464458 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 11:57:08.464458 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 11:57:08.464458 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 11:57:08.464458 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application May 19 11:57:08.465097 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application [None req-d7f4a902-7e18-405a-81d1-c3514f6226ea None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 11:57:08.465097 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 11:57:08.465097 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 11:57:08.465097 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 11:57:08.465097 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 11:57:08.465097 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 11:57:08.465097 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 11:57:08.465097 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 11:57:08.465097 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 11:57:08.465097 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 11:57:08.465097 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 11:57:08.465097 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 11:57:08.465097 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 11:57:08.465097 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 11:57:08.465097 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 11:57:08.465097 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 11:57:08.465876 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 11:57:08.465876 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 11:57:08.465876 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 11:57:08.465876 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 11:57:08.465876 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 11:57:08.465876 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 11:57:08.465876 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 11:57:08.465876 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 11:57:08.465876 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 11:57:08.465876 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.get_or_create( May 19 11:57:08.465876 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 11:57:08.465876 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application with Lock( May 19 11:57:08.465876 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 11:57:08.465876 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._enter() May 19 11:57:08.465876 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 11:57:08.465876 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 11:57:08.466648 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 11:57:08.466648 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.creator() May 19 11:57:08.466648 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 11:57:08.466648 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application created_value = creator( May 19 11:57:08.466648 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 11:57:08.466648 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 11:57:08.466648 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 11:57:08.466648 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 11:57:08.466648 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 11:57:08.466648 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 11:57:08.466648 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 11:57:08.466648 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application May 19 11:57:08.467203 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 58/113] 10.222.0.23 () {56 vars in 1096 bytes} [Thu May 19 11:57:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 11:57:08.467275 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 56/114] 10.222.0.23 () {56 vars in 1099 bytes} [Thu May 19 11:57:08 2022] GET /identity/v3/roles/member => generated 84 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 11:57:08.467275 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f4a902-7e18-405a-81d1-c3514f6226ea None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:57:08.469294 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783dd9c9-eb0e-4ca8-9852-f161ed0dddd4 None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:57:08.501208 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:57:08.562153 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:57:08.584095 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f4a902-7e18-405a-81d1-c3514f6226ea None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:57:08.585291 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783dd9c9-eb0e-4ca8-9852-f161ed0dddd4 None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:57:08.624938 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca64b25-df15-4450-8a04-a7bdb82ce91d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:57:08.625359 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567b889a-b8d2-4c7d-ba1a-8105275392f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:57:08.626306 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567b889a-b8d2-4c7d-ba1a-8105275392f3 None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:08.626527 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca64b25-df15-4450-8a04-a7bdb82ce91d None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:08.626819 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca64b25-df15-4450-8a04-a7bdb82ce91d None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:08.627102 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca64b25-df15-4450-8a04-a7bdb82ce91d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:08.627431 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567b889a-b8d2-4c7d-ba1a-8105275392f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:08.627935 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567b889a-b8d2-4c7d-ba1a-8105275392f3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:08.638002 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca64b25-df15-4450-8a04-a7bdb82ce91d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 11:57:08.647370 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application [None req-6ca64b25-df15-4450-8a04-a7bdb82ce91d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 11:57:08.647370 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 11:57:08.647370 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 11:57:08.647370 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 11:57:08.647370 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 11:57:08.647370 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 11:57:08.647370 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 11:57:08.647370 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 11:57:08.647370 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 11:57:08.647370 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 11:57:08.647370 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 11:57:08.647370 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 11:57:08.647370 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 11:57:08.647370 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 11:57:08.647370 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 11:57:08.647370 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 11:57:08.648373 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 11:57:08.648373 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 11:57:08.648373 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 11:57:08.648373 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 11:57:08.648373 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 11:57:08.648373 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 11:57:08.648373 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 11:57:08.648373 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 11:57:08.648373 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 11:57:08.648373 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.get_or_create( May 19 11:57:08.648373 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 11:57:08.648373 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application with Lock( May 19 11:57:08.648373 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 11:57:08.648373 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._enter() May 19 11:57:08.648373 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 11:57:08.648373 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 11:57:08.649337 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 11:57:08.649337 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.creator() May 19 11:57:08.649337 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 11:57:08.649337 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application created_value = creator( May 19 11:57:08.649337 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 11:57:08.649337 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 11:57:08.649337 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 11:57:08.649337 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 11:57:08.649337 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 11:57:08.649337 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 11:57:08.649337 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 11:57:08.649337 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application May 19 11:57:08.653323 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 57/115] 10.222.0.23 () {56 vars in 1114 bytes} [Thu May 19 11:57:08 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 11:57:08.654683 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325f4004-8488-4387-ba74-9c8c9eac9e0d None None] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:08.654981 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325f4004-8488-4387-ba74-9c8c9eac9e0d None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:08.655271 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325f4004-8488-4387-ba74-9c8c9eac9e0d None None] PATH_INFO: `/` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:08.657950 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 58/116] 10.222.0.23 () {54 vars in 926 bytes} [Thu May 19 11:57:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 11:57:08.658714 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325f4004-8488-4387-ba74-9c8c9eac9e0d None None] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:57:08.712197 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567b889a-b8d2-4c7d-ba1a-8105275392f3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:57:08.713994 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567b889a-b8d2-4c7d-ba1a-8105275392f3 None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:57:08.736958 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:57:08.786519 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325f4004-8488-4387-ba74-9c8c9eac9e0d None None] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:57:08.822998 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5abf62e2-2766-4c7c-bc11-c21f18445a46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:57:08.823592 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abf62e2-2766-4c7c-bc11-c21f18445a46 None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:08.823915 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abf62e2-2766-4c7c-bc11-c21f18445a46 None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:08.824201 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abf62e2-2766-4c7c-bc11-c21f18445a46 None admin] PATH_INFO: `/v3/roles` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:08.824964 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5abf62e2-2766-4c7c-bc11-c21f18445a46 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 11:57:08.825279 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5abf62e2-2766-4c7c-bc11-c21f18445a46 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:57:08.826682 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5abf62e2-2766-4c7c-bc11-c21f18445a46 None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:57:08.833754 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 59/117] 10.222.0.23 () {58 vars in 1282 bytes} [Thu May 19 11:57:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 11:57:08.834826 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567b889a-b8d2-4c7d-ba1a-8105275392f3 None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:57:08.840324 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 59/118] 10.222.0.23 () {56 vars in 1099 bytes} [Thu May 19 11:57:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:57:08.841212 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5abf62e2-2766-4c7c-bc11-c21f18445a46 None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:57:08.844742 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:57:08.957571 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567b889a-b8d2-4c7d-ba1a-8105275392f3 None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:57:08.960720 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5abf62e2-2766-4c7c-bc11-c21f18445a46 None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:57:08.978760 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:57:08.996677 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6792c728-d6db-4d2f-9bd3-d48cf136f3a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:57:08.997631 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6792c728-d6db-4d2f-9bd3-d48cf136f3a8 None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:08.997921 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6792c728-d6db-4d2f-9bd3-d48cf136f3a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:08.998203 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6792c728-d6db-4d2f-9bd3-d48cf136f3a8 None admin] PATH_INFO: `/v3/roles` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:08.999625 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6792c728-d6db-4d2f-9bd3-d48cf136f3a8 None admin] RBAC: Adding query filter params (name=member) {{(pid=80540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 11:57:08.999998 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6792c728-d6db-4d2f-9bd3-d48cf136f3a8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:57:09.002708 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6792c728-d6db-4d2f-9bd3-d48cf136f3a8 None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:57:09.008326 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4f8571-783f-43c0-b5f7-1f6a6dad4a02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:57:09.009018 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4f8571-783f-43c0-b5f7-1f6a6dad4a02 None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:09.009306 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4f8571-783f-43c0-b5f7-1f6a6dad4a02 None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:09.009583 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4f8571-783f-43c0-b5f7-1f6a6dad4a02 None admin] PATH_INFO: `/v3/roles` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:09.010339 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4f8571-783f-43c0-b5f7-1f6a6dad4a02 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 11:57:09.010661 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4f8571-783f-43c0-b5f7-1f6a6dad4a02 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:57:09.012058 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4f8571-783f-43c0-b5f7-1f6a6dad4a02 None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:57:09.013283 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 60/119] 10.222.0.23 () {56 vars in 1101 bytes} [Thu May 19 11:57:08 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:57:09.020189 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4604b794-daf2-4a54-b447-efa84b870673 None None] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:09.020924 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4604b794-daf2-4a54-b447-efa84b870673 None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:09.021307 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4604b794-daf2-4a54-b447-efa84b870673 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:09.030633 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 60/120] 10.222.0.23 () {56 vars in 1111 bytes} [Thu May 19 11:57:08 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:57:09.030633 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4f8571-783f-43c0-b5f7-1f6a6dad4a02 None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:57:09.038768 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.auth.core [None req-4604b794-daf2-4a54-b447-efa84b870673 None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 11:57:09.040031 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-4604b794-daf2-4a54-b447-efa84b870673 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 11:57:09.102143 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:57:09.119676 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 61/121] 10.222.0.23 () {58 vars in 1023 bytes} [Thu May 19 11:57:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 11:57:09.125303 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4604b794-daf2-4a54-b447-efa84b870673 None None] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:57:09.144886 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4f8571-783f-43c0-b5f7-1f6a6dad4a02 None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:57:09.186632 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70715ea7-1a68-4d03-bd2c-d2ce74a423e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:57:09.187610 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70715ea7-1a68-4d03-bd2c-d2ce74a423e1 None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:09.188252 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70715ea7-1a68-4d03-bd2c-d2ce74a423e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:09.188536 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70715ea7-1a68-4d03-bd2c-d2ce74a423e1 None admin] PATH_INFO: `/v3/projects/64092131887b4859ae47a1f4c3e0d24a` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:09.190456 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70715ea7-1a68-4d03-bd2c-d2ce74a423e1 None admin] RBAC: Authorizing `identity:get_project(project_id=64092131887b4859ae47a1f4c3e0d24a)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:57:09.192477 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70715ea7-1a68-4d03-bd2c-d2ce74a423e1 None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:57:09.199331 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 61/122] 10.222.0.23 () {56 vars in 1186 bytes} [Thu May 19 11:57:09 2022] GET /identity/v3/projects/64092131887b4859ae47a1f4c3e0d24a => generated 298 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:57:09.199331 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70715ea7-1a68-4d03-bd2c-d2ce74a423e1 None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:57:09.244401 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4604b794-daf2-4a54-b447-efa84b870673 None None] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:57:09.258384 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:57:09.280433 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97a5142-b44e-4af0-9897-3e2202617b4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:57:09.281961 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97a5142-b44e-4af0-9897-3e2202617b4c None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:09.282283 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97a5142-b44e-4af0-9897-3e2202617b4c None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:09.282650 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97a5142-b44e-4af0-9897-3e2202617b4c None admin] PATH_INFO: `/v3/users/11855e565a88475189d279de9469c271` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:09.283888 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97a5142-b44e-4af0-9897-3e2202617b4c None admin] RBAC: Authorizing `identity:get_user(user_id=11855e565a88475189d279de9469c271)` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:57:09.285384 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97a5142-b44e-4af0-9897-3e2202617b4c None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:57:09.287046 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 62/123] 10.222.0.23 () {56 vars in 1177 bytes} [Thu May 19 11:57:09 2022] GET /identity/v3/users/11855e565a88475189d279de9469c271 => generated 247 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:57:09.287851 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97a5142-b44e-4af0-9897-3e2202617b4c None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:57:09.325193 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70715ea7-1a68-4d03-bd2c-d2ce74a423e1 None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:57:09.359909 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:57:09.366392 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c10ef2-be0c-4ba8-b7ce-ac15e566440f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:57:09.367402 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c10ef2-be0c-4ba8-b7ce-ac15e566440f None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:09.367680 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c10ef2-be0c-4ba8-b7ce-ac15e566440f None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:09.368267 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c10ef2-be0c-4ba8-b7ce-ac15e566440f None admin] PATH_INFO: `/v3/users/267c35a521bb473ea2784646137bebda` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:09.376999 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c10ef2-be0c-4ba8-b7ce-ac15e566440f None admin] RBAC: Authorizing `identity:get_user(user_id=267c35a521bb473ea2784646137bebda)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:57:09.376999 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c10ef2-be0c-4ba8-b7ce-ac15e566440f None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:57:09.376999 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 62/124] 10.222.0.23 () {56 vars in 1177 bytes} [Thu May 19 11:57:09 2022] GET /identity/v3/users/267c35a521bb473ea2784646137bebda => generated 275 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:57:09.376999 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c10ef2-be0c-4ba8-b7ce-ac15e566440f None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:57:09.425667 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97a5142-b44e-4af0-9897-3e2202617b4c None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:57:09.470890 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f06ae1-83fb-4f33-9c81-a64cbbf26793 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:57:09.471781 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f06ae1-83fb-4f33-9c81-a64cbbf26793 None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:09.472096 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f06ae1-83fb-4f33-9c81-a64cbbf26793 None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:09.472380 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f06ae1-83fb-4f33-9c81-a64cbbf26793 None admin] PATH_INFO: `/v3/users/267c35a521bb473ea2784646137bebda` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:09.474332 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f06ae1-83fb-4f33-9c81-a64cbbf26793 None admin] RBAC: Authorizing `identity:get_user(user_id=267c35a521bb473ea2784646137bebda)` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:57:09.476173 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f06ae1-83fb-4f33-9c81-a64cbbf26793 None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:57:09.483710 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 63/125] 10.222.0.23 () {56 vars in 1177 bytes} [Thu May 19 11:57:09 2022] GET /identity/v3/users/267c35a521bb473ea2784646137bebda => generated 275 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:57:09.483710 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f06ae1-83fb-4f33-9c81-a64cbbf26793 None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:57:09.488456 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c10ef2-be0c-4ba8-b7ce-ac15e566440f None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:57:09.526182 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597434f1-7440-40c5-9ab0-48df051a8f98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:57:09.526988 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597434f1-7440-40c5-9ab0-48df051a8f98 None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:09.527278 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597434f1-7440-40c5-9ab0-48df051a8f98 None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:09.527560 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597434f1-7440-40c5-9ab0-48df051a8f98 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:09.539855 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597434f1-7440-40c5-9ab0-48df051a8f98 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 11:57:09.540206 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:57:09.548080 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application [None req-597434f1-7440-40c5-9ab0-48df051a8f98 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 11:57:09.548080 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 11:57:09.548080 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 11:57:09.548080 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 11:57:09.548080 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 11:57:09.548080 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 11:57:09.548080 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 11:57:09.548080 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 11:57:09.548080 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 11:57:09.548080 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 11:57:09.548080 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 11:57:09.548080 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 11:57:09.548080 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 11:57:09.548080 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 11:57:09.548080 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 11:57:09.548080 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 11:57:09.555287 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 11:57:09.555287 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 11:57:09.555287 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 11:57:09.555287 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 11:57:09.555287 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 11:57:09.555287 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 11:57:09.555287 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 11:57:09.555287 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 11:57:09.555287 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 11:57:09.555287 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.get_or_create( May 19 11:57:09.555287 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 11:57:09.555287 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application with Lock( May 19 11:57:09.555287 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 11:57:09.555287 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._enter() May 19 11:57:09.555287 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 11:57:09.555287 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 11:57:09.556115 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 11:57:09.556115 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.creator() May 19 11:57:09.556115 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 11:57:09.556115 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application created_value = creator( May 19 11:57:09.556115 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 11:57:09.556115 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 11:57:09.556115 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 11:57:09.556115 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 11:57:09.556115 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 11:57:09.556115 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 11:57:09.556115 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 11:57:09.556115 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application May 19 11:57:09.556115 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 63/126] 10.222.0.23 () {56 vars in 1099 bytes} [Thu May 19 11:57:09 2022] GET /identity/v3/roles/member => generated 84 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 11:57:09.556115 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597434f1-7440-40c5-9ab0-48df051a8f98 None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:57:09.615638 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f06ae1-83fb-4f33-9c81-a64cbbf26793 None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:57:09.623921 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:57:09.659636 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67bc2f9b-f494-4ab6-92b0-b71b59dba9e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:57:09.660501 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bc2f9b-f494-4ab6-92b0-b71b59dba9e5 None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:09.660794 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bc2f9b-f494-4ab6-92b0-b71b59dba9e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:09.661408 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bc2f9b-f494-4ab6-92b0-b71b59dba9e5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:09.662540 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67bc2f9b-f494-4ab6-92b0-b71b59dba9e5 None admin] RBAC: Adding query filter params (role.id=a49254f8009c463e88e5e0b3909c5904, scope.project.id=64092131887b4859ae47a1f4c3e0d24a, user.id=a92936fa37984cfda4b8a7a5cc363e6e) {{(pid=80540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 11:57:09.662853 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67bc2f9b-f494-4ab6-92b0-b71b59dba9e5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:57:09.665143 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67bc2f9b-f494-4ab6-92b0-b71b59dba9e5 None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:57:09.669524 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597434f1-7440-40c5-9ab0-48df051a8f98 None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:57:09.672537 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 64/127] 10.222.0.23 () {56 vars in 1374 bytes} [Thu May 19 11:57:09 2022] GET /identity/v3/role_assignments?user.id=a92936fa37984cfda4b8a7a5cc363e6e&scope.project.id=64092131887b4859ae47a1f4c3e0d24a&role.id=a49254f8009c463e88e5e0b3909c5904 => generated 259 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:57:09.673675 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67bc2f9b-f494-4ab6-92b0-b71b59dba9e5 None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:57:09.719072 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c05da4-c117-442b-910e-dbd0f498eb31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:57:09.720437 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c05da4-c117-442b-910e-dbd0f498eb31 None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:09.720924 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c05da4-c117-442b-910e-dbd0f498eb31 None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:09.721433 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c05da4-c117-442b-910e-dbd0f498eb31 None admin] PATH_INFO: `/v3/projects/64092131887b4859ae47a1f4c3e0d24a` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:09.723097 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c05da4-c117-442b-910e-dbd0f498eb31 None admin] RBAC: Authorizing `identity:get_project(project_id=64092131887b4859ae47a1f4c3e0d24a)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:57:09.725104 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c05da4-c117-442b-910e-dbd0f498eb31 None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:57:09.727344 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 64/128] 10.222.0.23 () {56 vars in 1186 bytes} [Thu May 19 11:57:09 2022] GET /identity/v3/projects/64092131887b4859ae47a1f4c3e0d24a => generated 298 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:57:09.733471 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c05da4-c117-442b-910e-dbd0f498eb31 None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:57:09.801494 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67bc2f9b-f494-4ab6-92b0-b71b59dba9e5 None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:57:09.818937 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:57:09.846475 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d868538-01af-4c80-af2c-f0c1ab78c977 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:57:09.847758 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d868538-01af-4c80-af2c-f0c1ab78c977 None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:09.848021 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d868538-01af-4c80-af2c-f0c1ab78c977 None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:09.848537 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d868538-01af-4c80-af2c-f0c1ab78c977 None admin] PATH_INFO: `/v3/projects/64092131887b4859ae47a1f4c3e0d24a` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:09.851032 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d868538-01af-4c80-af2c-f0c1ab78c977 None admin] RBAC: Authorizing `identity:get_project(project_id=64092131887b4859ae47a1f4c3e0d24a)` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:57:09.853562 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d868538-01af-4c80-af2c-f0c1ab78c977 None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:57:09.861765 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 65/129] 10.222.0.23 () {56 vars in 1186 bytes} [Thu May 19 11:57:09 2022] GET /identity/v3/projects/64092131887b4859ae47a1f4c3e0d24a => generated 298 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:57:09.862865 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d868538-01af-4c80-af2c-f0c1ab78c977 None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:57:09.865574 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c05da4-c117-442b-910e-dbd0f498eb31 None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:57:09.896938 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:57:09.905795 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014b163d-36db-4ce0-8ed8-00bbab98cb0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:57:09.907201 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014b163d-36db-4ce0-8ed8-00bbab98cb0e None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:09.910538 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014b163d-36db-4ce0-8ed8-00bbab98cb0e None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:09.910538 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014b163d-36db-4ce0-8ed8-00bbab98cb0e None admin] PATH_INFO: `/v3/projects/64092131887b4859ae47a1f4c3e0d24a` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:09.910538 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014b163d-36db-4ce0-8ed8-00bbab98cb0e None admin] RBAC: Authorizing `identity:get_project(project_id=64092131887b4859ae47a1f4c3e0d24a)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:57:09.910923 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014b163d-36db-4ce0-8ed8-00bbab98cb0e None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:57:09.912883 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 65/130] 10.222.0.23 () {56 vars in 1186 bytes} [Thu May 19 11:57:09 2022] GET /identity/v3/projects/64092131887b4859ae47a1f4c3e0d24a => generated 298 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:57:09.913441 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014b163d-36db-4ce0-8ed8-00bbab98cb0e None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:57:10.001693 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d868538-01af-4c80-af2c-f0c1ab78c977 None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:57:10.037840 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014b163d-36db-4ce0-8ed8-00bbab98cb0e None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:57:10.050995 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443c3ce9-a321-429b-9f27-7398b0654795 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:57:10.052573 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443c3ce9-a321-429b-9f27-7398b0654795 None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:10.053015 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443c3ce9-a321-429b-9f27-7398b0654795 None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:10.053675 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443c3ce9-a321-429b-9f27-7398b0654795 None admin] PATH_INFO: `/v3/roles` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:10.055090 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443c3ce9-a321-429b-9f27-7398b0654795 None admin] RBAC: Adding query filter params (name=member) {{(pid=80540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 11:57:10.055840 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443c3ce9-a321-429b-9f27-7398b0654795 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:57:10.058112 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443c3ce9-a321-429b-9f27-7398b0654795 None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:57:10.070815 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 66/131] 10.222.0.23 () {56 vars in 1101 bytes} [Thu May 19 11:57:09 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:57:10.072562 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443c3ce9-a321-429b-9f27-7398b0654795 None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:57:10.077272 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177cc5cb-109e-4c61-9824-eb477df4e53d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:57:10.078088 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177cc5cb-109e-4c61-9824-eb477df4e53d None admin] REQUEST_METHOD: `PUT` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:10.078576 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177cc5cb-109e-4c61-9824-eb477df4e53d None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:10.078864 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177cc5cb-109e-4c61-9824-eb477df4e53d None admin] PATH_INFO: `/v3/projects/64092131887b4859ae47a1f4c3e0d24a/users/11855e565a88475189d279de9469c271/roles/9db5671c65d3414ea2682afa2ccf1403` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:10.080941 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177cc5cb-109e-4c61-9824-eb477df4e53d None admin] RBAC: Authorizing `identity:create_grant(project_id=64092131887b4859ae47a1f4c3e0d24a, user_id=11855e565a88475189d279de9469c271, role_id=9db5671c65d3414ea2682afa2ccf1403)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:57:10.082462 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177cc5cb-109e-4c61-9824-eb477df4e53d None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:57:10.122753 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:57:10.184070 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 66/132] 10.222.0.23 () {58 vars in 1439 bytes} [Thu May 19 11:57:09 2022] PUT /identity/v3/projects/64092131887b4859ae47a1f4c3e0d24a/users/11855e565a88475189d279de9469c271/roles/9db5671c65d3414ea2682afa2ccf1403 => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 11:57:10.184070 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177cc5cb-109e-4c61-9824-eb477df4e53d None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:57:10.206934 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443c3ce9-a321-429b-9f27-7398b0654795 None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:57:10.245457 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d39141d-353e-40b4-8bfa-7e3a45798582 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:57:10.246364 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d39141d-353e-40b4-8bfa-7e3a45798582 None admin] REQUEST_METHOD: `PUT` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:10.246648 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d39141d-353e-40b4-8bfa-7e3a45798582 None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:10.246930 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d39141d-353e-40b4-8bfa-7e3a45798582 None admin] PATH_INFO: `/v3/projects/64092131887b4859ae47a1f4c3e0d24a/users/267c35a521bb473ea2784646137bebda/roles/259eb400cdb4439db72d14456ae9cd08` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:10.249293 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d39141d-353e-40b4-8bfa-7e3a45798582 None admin] RBAC: Authorizing `identity:create_grant(project_id=64092131887b4859ae47a1f4c3e0d24a, user_id=267c35a521bb473ea2784646137bebda, role_id=259eb400cdb4439db72d14456ae9cd08)` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:57:10.251224 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d39141d-353e-40b4-8bfa-7e3a45798582 None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:57:10.268279 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:57:10.333321 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177cc5cb-109e-4c61-9824-eb477df4e53d None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:57:10.374496 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2220e96a-3e23-4485-a647-babbfbd48ab0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:57:10.376257 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2220e96a-3e23-4485-a647-babbfbd48ab0 None admin] REQUEST_METHOD: `PUT` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:10.376761 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2220e96a-3e23-4485-a647-babbfbd48ab0 None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:10.377286 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2220e96a-3e23-4485-a647-babbfbd48ab0 None admin] PATH_INFO: `/v3/projects/64092131887b4859ae47a1f4c3e0d24a/users/267c35a521bb473ea2784646137bebda/roles/8215583bb4224c75b093ae8e3993eb9f` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:10.377872 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 67/133] 10.222.0.23 () {58 vars in 1439 bytes} [Thu May 19 11:57:10 2022] PUT /identity/v3/projects/64092131887b4859ae47a1f4c3e0d24a/users/267c35a521bb473ea2784646137bebda/roles/259eb400cdb4439db72d14456ae9cd08 => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 11:57:10.378508 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d39141d-353e-40b4-8bfa-7e3a45798582 None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:57:10.380817 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2220e96a-3e23-4485-a647-babbfbd48ab0 None admin] RBAC: Authorizing `identity:create_grant(project_id=64092131887b4859ae47a1f4c3e0d24a, user_id=267c35a521bb473ea2784646137bebda, role_id=8215583bb4224c75b093ae8e3993eb9f)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:57:10.382617 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2220e96a-3e23-4485-a647-babbfbd48ab0 None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:57:10.468281 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 67/134] 10.222.0.23 () {58 vars in 1439 bytes} [Thu May 19 11:57:10 2022] PUT /identity/v3/projects/64092131887b4859ae47a1f4c3e0d24a/users/267c35a521bb473ea2784646137bebda/roles/8215583bb4224c75b093ae8e3993eb9f => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 11:57:10.470993 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0f909d-fc5e-417e-9a2d-279f90ccb519 None None] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:10.480037 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0f909d-fc5e-417e-9a2d-279f90ccb519 None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:10.480742 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0f909d-fc5e-417e-9a2d-279f90ccb519 None None] PATH_INFO: `/` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:10.482404 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 68/135] 10.222.0.23 () {54 vars in 926 bytes} [Thu May 19 11:57:10 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 11:57:10.491420 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b44470-21ee-4331-83fd-15333a237c9b None None] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:10.491852 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b44470-21ee-4331-83fd-15333a237c9b None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:10.492139 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b44470-21ee-4331-83fd-15333a237c9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:10.515418 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.auth.core [None req-33b44470-21ee-4331-83fd-15333a237c9b None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 11:57:10.517088 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-33b44470-21ee-4331-83fd-15333a237c9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 11:57:10.558194 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d39141d-353e-40b4-8bfa-7e3a45798582 None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:57:10.566957 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:57:10.610924 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9e4e5b-4810-433b-a321-d7f9ce409420 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:57:10.611957 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9e4e5b-4810-433b-a321-d7f9ce409420 None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:10.612255 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9e4e5b-4810-433b-a321-d7f9ce409420 None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:10.612538 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9e4e5b-4810-433b-a321-d7f9ce409420 None admin] PATH_INFO: `/v3/users/267c35a521bb473ea2784646137bebda` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:10.616487 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9e4e5b-4810-433b-a321-d7f9ce409420 None admin] RBAC: Authorizing `identity:get_user(user_id=267c35a521bb473ea2784646137bebda)` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:57:10.618071 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9e4e5b-4810-433b-a321-d7f9ce409420 None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:57:10.621883 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 68/136] 10.222.0.23 () {56 vars in 1177 bytes} [Thu May 19 11:57:10 2022] GET /identity/v3/users/267c35a521bb473ea2784646137bebda => generated 275 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:57:10.625706 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9e4e5b-4810-433b-a321-d7f9ce409420 None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:57:10.655252 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 69/137] 10.222.0.23 () {58 vars in 1023 bytes} [Thu May 19 11:57:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 11:57:10.703264 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b44470-21ee-4331-83fd-15333a237c9b None None] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:57:10.780843 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:57:10.796934 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9e4e5b-4810-433b-a321-d7f9ce409420 None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:57:10.848706 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92aff911-6b6e-43eb-880d-baeca8bbb446 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:57:10.849771 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92aff911-6b6e-43eb-880d-baeca8bbb446 None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:10.850562 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92aff911-6b6e-43eb-880d-baeca8bbb446 None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:10.850861 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92aff911-6b6e-43eb-880d-baeca8bbb446 None admin] PATH_INFO: `/v3/projects/4aeddbfe954d472aa273e1df1efca1fd` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:10.852779 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92aff911-6b6e-43eb-880d-baeca8bbb446 None admin] RBAC: Authorizing `identity:get_project(project_id=4aeddbfe954d472aa273e1df1efca1fd)` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:57:10.854804 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92aff911-6b6e-43eb-880d-baeca8bbb446 None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:57:10.856932 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b44470-21ee-4331-83fd-15333a237c9b None None] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:57:10.857204 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 69/138] 10.222.0.23 () {56 vars in 1186 bytes} [Thu May 19 11:57:10 2022] GET /identity/v3/projects/4aeddbfe954d472aa273e1df1efca1fd => generated 312 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:57:10.862953 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92aff911-6b6e-43eb-880d-baeca8bbb446 None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:57:10.919690 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfc6d12-86ce-4793-b796-ad10315ced39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:57:10.922448 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfc6d12-86ce-4793-b796-ad10315ced39 None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:10.922723 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfc6d12-86ce-4793-b796-ad10315ced39 None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:10.923005 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfc6d12-86ce-4793-b796-ad10315ced39 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:10.925980 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dfc6d12-86ce-4793-b796-ad10315ced39 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:57:10.927661 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dfc6d12-86ce-4793-b796-ad10315ced39 None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:57:10.930711 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 70/139] 10.222.0.23 () {56 vars in 1108 bytes} [Thu May 19 11:57:10 2022] GET /identity/v3/domains/default => generated 199 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 11:57:10.937191 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfc6d12-86ce-4793-b796-ad10315ced39 None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:57:10.962679 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:57:11.111742 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92aff911-6b6e-43eb-880d-baeca8bbb446 None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:57:11.168255 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfc6d12-86ce-4793-b796-ad10315ced39 None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:57:11.169670 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0772282-e4f0-419e-9a02-861670bf6d00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:57:11.170859 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0772282-e4f0-419e-9a02-861670bf6d00 None admin] REQUEST_METHOD: `PUT` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:11.171376 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0772282-e4f0-419e-9a02-861670bf6d00 None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:11.171920 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0772282-e4f0-419e-9a02-861670bf6d00 None admin] PATH_INFO: `/v3/projects/4aeddbfe954d472aa273e1df1efca1fd/users/267c35a521bb473ea2784646137bebda/roles/259eb400cdb4439db72d14456ae9cd08` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:11.174784 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0772282-e4f0-419e-9a02-861670bf6d00 None admin] RBAC: Authorizing `identity:create_grant(project_id=4aeddbfe954d472aa273e1df1efca1fd, user_id=267c35a521bb473ea2784646137bebda, role_id=259eb400cdb4439db72d14456ae9cd08)` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:57:11.176562 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0772282-e4f0-419e-9a02-861670bf6d00 None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:57:11.191979 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:57:11.215154 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f8d161-1667-4bfa-af8e-147d6d58b1c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:57:11.216262 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f8d161-1667-4bfa-af8e-147d6d58b1c0 None admin] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:11.216738 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f8d161-1667-4bfa-af8e-147d6d58b1c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:11.217265 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f8d161-1667-4bfa-af8e-147d6d58b1c0 None admin] PATH_INFO: `/v3/users` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:11.218205 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f8d161-1667-4bfa-af8e-147d6d58b1c0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:57:11.220926 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f8d161-1667-4bfa-af8e-147d6d58b1c0 None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:57:11.277467 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 70/140] 10.222.0.23 () {58 vars in 1439 bytes} [Thu May 19 11:57:10 2022] PUT /identity/v3/projects/4aeddbfe954d472aa273e1df1efca1fd/users/267c35a521bb473ea2784646137bebda/roles/259eb400cdb4439db72d14456ae9cd08 => generated 0 bytes in 415 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 11:57:11.368750 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 71/141] 10.222.0.23 () {60 vars in 1132 bytes} [Thu May 19 11:57:10 2022] POST /identity/v3/users => generated 283 bytes in 432 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 11:57:12.449628 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1204320e-8d58-41da-825c-6a41a50852a0 None None] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:12.450695 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1204320e-8d58-41da-825c-6a41a50852a0 None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:12.451196 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1204320e-8d58-41da-825c-6a41a50852a0 None None] PATH_INFO: `/` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:12.453740 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 71/142] 10.222.0.23 () {54 vars in 926 bytes} [Thu May 19 11:57:12 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 11:57:12.459068 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90be51fd-aa04-47c5-8839-25e2dee6f195 None None] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:12.459999 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90be51fd-aa04-47c5-8839-25e2dee6f195 None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:12.460488 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90be51fd-aa04-47c5-8839-25e2dee6f195 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:12.485887 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.auth.core [None req-90be51fd-aa04-47c5-8839-25e2dee6f195 None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 11:57:12.487530 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-90be51fd-aa04-47c5-8839-25e2dee6f195 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 11:57:12.563319 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 72/143] 10.222.0.23 () {58 vars in 1023 bytes} [Thu May 19 11:57:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 11:57:12.597460 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1204320e-8d58-41da-825c-6a41a50852a0 None None] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:57:12.737879 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:57:12.744529 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1204320e-8d58-41da-825c-6a41a50852a0 None None] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:57:12.789825 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4c060b-9158-4b80-a0e4-d0c46a5de09a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:57:12.790638 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4c060b-9158-4b80-a0e4-d0c46a5de09a None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:12.790925 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4c060b-9158-4b80-a0e4-d0c46a5de09a None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:12.791205 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4c060b-9158-4b80-a0e4-d0c46a5de09a None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:12.808295 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4c060b-9158-4b80-a0e4-d0c46a5de09a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 11:57:12.818634 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application [None req-7a4c060b-9158-4b80-a0e4-d0c46a5de09a None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 19 11:57:12.818634 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 11:57:12.818634 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 11:57:12.818634 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 11:57:12.818634 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 11:57:12.818634 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 11:57:12.818634 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 11:57:12.818634 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 11:57:12.818634 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 11:57:12.818634 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 11:57:12.818634 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 11:57:12.818634 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 11:57:12.818634 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 11:57:12.818634 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 11:57:12.818634 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 11:57:12.818634 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 11:57:12.819459 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 11:57:12.819459 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 11:57:12.819459 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 11:57:12.819459 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 11:57:12.819459 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 11:57:12.819459 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 11:57:12.819459 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 11:57:12.819459 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 11:57:12.819459 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 11:57:12.819459 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.get_or_create( May 19 11:57:12.819459 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 11:57:12.819459 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application with Lock( May 19 11:57:12.819459 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 11:57:12.819459 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._enter() May 19 11:57:12.819459 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 11:57:12.819459 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 11:57:12.820507 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 11:57:12.820507 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.creator() May 19 11:57:12.820507 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 11:57:12.820507 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application created_value = creator( May 19 11:57:12.820507 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 11:57:12.820507 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 11:57:12.820507 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 11:57:12.820507 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 11:57:12.820507 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 11:57:12.820507 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 11:57:12.820507 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 19 11:57:12.820507 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application May 19 11:57:12.824802 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 72/144] 10.222.0.23 () {56 vars in 1099 bytes} [Thu May 19 11:57:12 2022] GET /identity/v3/roles/reader => generated 84 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 11:57:12.831242 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90be51fd-aa04-47c5-8839-25e2dee6f195 None None] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:57:12.961569 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:57:12.965187 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90be51fd-aa04-47c5-8839-25e2dee6f195 None None] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:57:13.010924 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942d55c5-3f89-4f1a-b843-aa9f2f7e0b38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:57:13.011519 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942d55c5-3f89-4f1a-b843-aa9f2f7e0b38 None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:13.011828 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942d55c5-3f89-4f1a-b843-aa9f2f7e0b38 None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:13.012118 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942d55c5-3f89-4f1a-b843-aa9f2f7e0b38 None admin] PATH_INFO: `/v3/roles` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:13.012900 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942d55c5-3f89-4f1a-b843-aa9f2f7e0b38 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 11:57:13.013210 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942d55c5-3f89-4f1a-b843-aa9f2f7e0b38 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:57:13.014841 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942d55c5-3f89-4f1a-b843-aa9f2f7e0b38 None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:57:13.030762 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 73/145] 10.222.0.23 () {56 vars in 1101 bytes} [Thu May 19 11:57:12 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:57:13.037285 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4c060b-9158-4b80-a0e4-d0c46a5de09a None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:57:13.188766 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4c060b-9158-4b80-a0e4-d0c46a5de09a None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:57:13.237336 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e45558-b3c3-4669-abb4-b9e2212e226c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:57:13.238227 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e45558-b3c3-4669-abb4-b9e2212e226c None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:13.238513 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e45558-b3c3-4669-abb4-b9e2212e226c None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:13.238861 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e45558-b3c3-4669-abb4-b9e2212e226c None admin] PATH_INFO: `/v3/users/a92936fa37984cfda4b8a7a5cc363e6e` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:13.244558 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e45558-b3c3-4669-abb4-b9e2212e226c None admin] RBAC: Authorizing `identity:get_user(user_id=a92936fa37984cfda4b8a7a5cc363e6e)` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:57:13.246068 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e45558-b3c3-4669-abb4-b9e2212e226c None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:57:13.247974 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 73/146] 10.222.0.23 () {56 vars in 1177 bytes} [Thu May 19 11:57:13 2022] GET /identity/v3/users/a92936fa37984cfda4b8a7a5cc363e6e => generated 289 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:57:13.253839 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942d55c5-3f89-4f1a-b843-aa9f2f7e0b38 None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:57:13.310020 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414b87d7-cbe2-4fd7-aa43-50fd865f721a None None] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:13.310324 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414b87d7-cbe2-4fd7-aa43-50fd865f721a None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:13.310593 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414b87d7-cbe2-4fd7-aa43-50fd865f721a None None] PATH_INFO: `/` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:13.312121 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 74/147] 10.222.0.23 () {54 vars in 926 bytes} [Thu May 19 11:57:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 11:57:13.318109 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3176973c-0dd1-4087-ae1f-85ffcb39d20d None None] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:13.318402 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3176973c-0dd1-4087-ae1f-85ffcb39d20d None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:13.318687 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3176973c-0dd1-4087-ae1f-85ffcb39d20d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:13.352924 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.auth.core [None req-3176973c-0dd1-4087-ae1f-85ffcb39d20d None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 11:57:13.354155 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-3176973c-0dd1-4087-ae1f-85ffcb39d20d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 11:57:13.396207 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:57:13.460723 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942d55c5-3f89-4f1a-b843-aa9f2f7e0b38 None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:57:13.512542 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5039c7b-7f35-4294-b93a-32e20a3befc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:57:13.513815 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5039c7b-7f35-4294-b93a-32e20a3befc1 None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:13.514304 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5039c7b-7f35-4294-b93a-32e20a3befc1 None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:13.514933 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5039c7b-7f35-4294-b93a-32e20a3befc1 None admin] PATH_INFO: `/v3/projects/64092131887b4859ae47a1f4c3e0d24a` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:13.520597 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5039c7b-7f35-4294-b93a-32e20a3befc1 None admin] RBAC: Authorizing `identity:get_project(project_id=64092131887b4859ae47a1f4c3e0d24a)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:57:13.522348 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5039c7b-7f35-4294-b93a-32e20a3befc1 None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:57:13.524379 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 74/148] 10.222.0.23 () {56 vars in 1186 bytes} [Thu May 19 11:57:13 2022] GET /identity/v3/projects/64092131887b4859ae47a1f4c3e0d24a => generated 298 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:57:13.529404 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 75/149] 10.222.0.23 () {58 vars in 1023 bytes} [Thu May 19 11:57:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 11:57:13.530705 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5039c7b-7f35-4294-b93a-32e20a3befc1 None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:57:13.611012 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3176973c-0dd1-4087-ae1f-85ffcb39d20d None None] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:57:13.614343 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:57:13.676298 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5039c7b-7f35-4294-b93a-32e20a3befc1 None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:57:13.740691 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22d6253-71d4-458f-a9fa-29adf8e28ff8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:57:13.742090 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22d6253-71d4-458f-a9fa-29adf8e28ff8 None admin] REQUEST_METHOD: `PUT` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:13.742610 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22d6253-71d4-458f-a9fa-29adf8e28ff8 None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:13.743087 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22d6253-71d4-458f-a9fa-29adf8e28ff8 None admin] PATH_INFO: `/v3/projects/64092131887b4859ae47a1f4c3e0d24a/users/a92936fa37984cfda4b8a7a5cc363e6e/roles/a49254f8009c463e88e5e0b3909c5904` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:13.745661 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22d6253-71d4-458f-a9fa-29adf8e28ff8 None admin] RBAC: Authorizing `identity:create_grant(project_id=64092131887b4859ae47a1f4c3e0d24a, user_id=a92936fa37984cfda4b8a7a5cc363e6e, role_id=a49254f8009c463e88e5e0b3909c5904)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:57:13.747428 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22d6253-71d4-458f-a9fa-29adf8e28ff8 None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:57:13.800034 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3176973c-0dd1-4087-ae1f-85ffcb39d20d None None] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:57:13.839374 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 75/150] 10.222.0.23 () {58 vars in 1439 bytes} [Thu May 19 11:57:13 2022] PUT /identity/v3/projects/64092131887b4859ae47a1f4c3e0d24a/users/a92936fa37984cfda4b8a7a5cc363e6e/roles/a49254f8009c463e88e5e0b3909c5904 => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 11:57:13.855145 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc18e03-c94c-4aa9-970b-a708576210da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:57:13.856000 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc18e03-c94c-4aa9-970b-a708576210da None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:13.856290 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc18e03-c94c-4aa9-970b-a708576210da None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:13.856568 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc18e03-c94c-4aa9-970b-a708576210da None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:13.874113 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc18e03-c94c-4aa9-970b-a708576210da None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 11:57:13.882512 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application [None req-abc18e03-c94c-4aa9-970b-a708576210da None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 11:57:13.882512 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 11:57:13.882512 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 11:57:13.882512 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 11:57:13.882512 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 11:57:13.882512 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 11:57:13.882512 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 11:57:13.882512 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 11:57:13.882512 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 11:57:13.882512 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 11:57:13.882512 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 11:57:13.882512 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 11:57:13.882512 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 11:57:13.882512 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 11:57:13.882512 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 11:57:13.882512 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 11:57:13.883418 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 11:57:13.883418 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 11:57:13.883418 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 11:57:13.883418 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 11:57:13.883418 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 11:57:13.883418 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 11:57:13.883418 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 11:57:13.883418 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 11:57:13.883418 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 11:57:13.883418 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.get_or_create( May 19 11:57:13.883418 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 11:57:13.883418 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application with Lock( May 19 11:57:13.883418 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 11:57:13.883418 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._enter() May 19 11:57:13.883418 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 11:57:13.883418 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 11:57:13.884197 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 11:57:13.884197 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.creator() May 19 11:57:13.884197 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 11:57:13.884197 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application created_value = creator( May 19 11:57:13.884197 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 11:57:13.884197 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 11:57:13.884197 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 11:57:13.884197 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 11:57:13.884197 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 11:57:13.884197 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 11:57:13.884197 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 11:57:13.884197 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application May 19 11:57:13.888183 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 76/151] 10.222.0.23 () {56 vars in 1096 bytes} [Thu May 19 11:57:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 11:57:13.889977 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22d6253-71d4-458f-a9fa-29adf8e28ff8 None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:57:14.040185 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:57:14.054506 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22d6253-71d4-458f-a9fa-29adf8e28ff8 None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:57:14.106137 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ba46f4-99a7-46eb-a8a9-e47cec9dd466 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:57:14.106860 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ba46f4-99a7-46eb-a8a9-e47cec9dd466 None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:14.107156 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ba46f4-99a7-46eb-a8a9-e47cec9dd466 None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:14.107436 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ba46f4-99a7-46eb-a8a9-e47cec9dd466 None admin] PATH_INFO: `/v3/roles` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:14.108277 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ba46f4-99a7-46eb-a8a9-e47cec9dd466 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 11:57:14.108589 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ba46f4-99a7-46eb-a8a9-e47cec9dd466 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:57:14.110029 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ba46f4-99a7-46eb-a8a9-e47cec9dd466 None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:57:14.124224 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 76/152] 10.222.0.23 () {56 vars in 1099 bytes} [Thu May 19 11:57:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:57:14.126375 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc18e03-c94c-4aa9-970b-a708576210da None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:57:14.174501 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:57:14.298016 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc18e03-c94c-4aa9-970b-a708576210da None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:57:14.343555 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e70fe2-a1ae-4ab4-9065-e5b0c9f4baad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:57:14.344647 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e70fe2-a1ae-4ab4-9065-e5b0c9f4baad None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:14.344927 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e70fe2-a1ae-4ab4-9065-e5b0c9f4baad None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:14.345205 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e70fe2-a1ae-4ab4-9065-e5b0c9f4baad None admin] PATH_INFO: `/v3/users/11855e565a88475189d279de9469c271` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:14.346461 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e70fe2-a1ae-4ab4-9065-e5b0c9f4baad None admin] RBAC: Authorizing `identity:get_user(user_id=11855e565a88475189d279de9469c271)` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:57:14.348046 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e70fe2-a1ae-4ab4-9065-e5b0c9f4baad None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:57:14.350018 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 77/153] 10.222.0.23 () {56 vars in 1177 bytes} [Thu May 19 11:57:14 2022] GET /identity/v3/users/11855e565a88475189d279de9469c271 => generated 247 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:57:14.355571 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ba46f4-99a7-46eb-a8a9-e47cec9dd466 None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:57:14.481388 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ba46f4-99a7-46eb-a8a9-e47cec9dd466 None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:57:14.534014 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68acf714-6a71-4cc5-b565-20595d6fbffb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:57:14.534689 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68acf714-6a71-4cc5-b565-20595d6fbffb None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:14.534973 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68acf714-6a71-4cc5-b565-20595d6fbffb None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:14.535246 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68acf714-6a71-4cc5-b565-20595d6fbffb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:14.564485 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:57:14.572170 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47d1bbb-4b4f-41b7-9806-683bb56d2a82 None None] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:14.572692 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47d1bbb-4b4f-41b7-9806-683bb56d2a82 None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:14.572838 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47d1bbb-4b4f-41b7-9806-683bb56d2a82 None None] PATH_INFO: `/` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:14.574202 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 78/154] 10.222.0.23 () {54 vars in 926 bytes} [Thu May 19 11:57:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 11:57:14.582393 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031d968e-8052-4df7-880f-5d00b7063937 None None] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:14.582704 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031d968e-8052-4df7-880f-5d00b7063937 None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:14.582919 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031d968e-8052-4df7-880f-5d00b7063937 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:14.609002 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.auth.core [None req-031d968e-8052-4df7-880f-5d00b7063937 None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 11:57:14.610299 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-031d968e-8052-4df7-880f-5d00b7063937 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 11:57:14.705110 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 79/155] 10.222.0.23 () {58 vars in 1023 bytes} [Thu May 19 11:57:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 11:57:14.708523 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68acf714-6a71-4cc5-b565-20595d6fbffb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:57:14.710285 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68acf714-6a71-4cc5-b565-20595d6fbffb None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:57:14.731291 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031d968e-8052-4df7-880f-5d00b7063937 None None] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:57:14.756241 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:57:14.862279 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031d968e-8052-4df7-880f-5d00b7063937 None None] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:57:14.904414 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f07059d-0960-4022-a369-38c350887b5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:57:14.905223 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f07059d-0960-4022-a369-38c350887b5a None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:14.905507 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f07059d-0960-4022-a369-38c350887b5a None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:14.905783 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f07059d-0960-4022-a369-38c350887b5a None admin] PATH_INFO: `/v3/roles/member` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:14.917832 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f07059d-0960-4022-a369-38c350887b5a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 11:57:14.926975 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application [None req-5f07059d-0960-4022-a369-38c350887b5a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 11:57:14.926975 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 11:57:14.926975 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 11:57:14.926975 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 11:57:14.926975 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 11:57:14.926975 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 11:57:14.926975 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 11:57:14.926975 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 11:57:14.926975 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 11:57:14.926975 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 11:57:14.926975 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 11:57:14.926975 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 11:57:14.926975 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 11:57:14.926975 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 11:57:14.926975 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 11:57:14.926975 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 11:57:14.928112 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 11:57:14.928112 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 11:57:14.928112 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 11:57:14.928112 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 11:57:14.928112 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 11:57:14.928112 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 11:57:14.928112 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 11:57:14.928112 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 11:57:14.928112 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 11:57:14.928112 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.get_or_create( May 19 11:57:14.928112 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 11:57:14.928112 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application with Lock( May 19 11:57:14.928112 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 11:57:14.928112 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._enter() May 19 11:57:14.928112 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 11:57:14.928112 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 11:57:14.928876 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 11:57:14.928876 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.creator() May 19 11:57:14.928876 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 11:57:14.928876 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application created_value = creator( May 19 11:57:14.928876 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 11:57:14.928876 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 11:57:14.928876 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 11:57:14.928876 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 11:57:14.928876 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 11:57:14.928876 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 11:57:14.928876 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 11:57:14.928876 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application May 19 11:57:14.928876 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 80/156] 10.222.0.23 () {56 vars in 1099 bytes} [Thu May 19 11:57:14 2022] GET /identity/v3/roles/member => generated 84 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 11:57:14.935407 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f07059d-0960-4022-a369-38c350887b5a None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:57:14.979435 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 77/157] 10.222.0.23 () {58 vars in 1282 bytes} [Thu May 19 11:57:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 624 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 11:57:14.985167 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68acf714-6a71-4cc5-b565-20595d6fbffb None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:57:15.064376 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:57:15.089728 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:57:15.113993 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f07059d-0960-4022-a369-38c350887b5a None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:57:15.176400 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87867bc-4501-4418-bfc1-bdb49ad9b1a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:57:15.177584 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87867bc-4501-4418-bfc1-bdb49ad9b1a8 None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:15.178072 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87867bc-4501-4418-bfc1-bdb49ad9b1a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:15.178569 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87867bc-4501-4418-bfc1-bdb49ad9b1a8 None admin] PATH_INFO: `/v3/roles` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:15.179554 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87867bc-4501-4418-bfc1-bdb49ad9b1a8 None admin] RBAC: Adding query filter params (name=member) {{(pid=80540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 11:57:15.180128 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87867bc-4501-4418-bfc1-bdb49ad9b1a8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:57:15.182658 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87867bc-4501-4418-bfc1-bdb49ad9b1a8 None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:57:15.184292 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68acf714-6a71-4cc5-b565-20595d6fbffb None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:57:15.195346 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 81/158] 10.222.0.23 () {56 vars in 1101 bytes} [Thu May 19 11:57:14 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:57:15.197680 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ad48ff-008e-4a2b-8a7d-329db8a362c8 None None] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:15.198282 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ad48ff-008e-4a2b-8a7d-329db8a362c8 None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:15.198755 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ad48ff-008e-4a2b-8a7d-329db8a362c8 None None] PATH_INFO: `/` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:15.200438 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 82/159] 10.222.0.23 () {54 vars in 926 bytes} [Thu May 19 11:57:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 11:57:15.202010 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ad48ff-008e-4a2b-8a7d-329db8a362c8 None None] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:57:15.241472 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c9efcf-f6da-4133-a8d5-6f0601ec97c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:57:15.242414 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c9efcf-f6da-4133-a8d5-6f0601ec97c4 None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:15.242711 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c9efcf-f6da-4133-a8d5-6f0601ec97c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:15.243103 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c9efcf-f6da-4133-a8d5-6f0601ec97c4 None admin] PATH_INFO: `/v3/projects/64092131887b4859ae47a1f4c3e0d24a` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:15.244531 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c9efcf-f6da-4133-a8d5-6f0601ec97c4 None admin] RBAC: Authorizing `identity:get_project(project_id=64092131887b4859ae47a1f4c3e0d24a)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:57:15.246146 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c9efcf-f6da-4133-a8d5-6f0601ec97c4 None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:57:15.248439 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 78/160] 10.222.0.23 () {56 vars in 1186 bytes} [Thu May 19 11:57:14 2022] GET /identity/v3/projects/64092131887b4859ae47a1f4c3e0d24a => generated 298 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 11:57:15.249772 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48523629-fa14-42b2-9d77-96ce8f7075b3 None None] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:15.250269 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48523629-fa14-42b2-9d77-96ce8f7075b3 None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:15.250677 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48523629-fa14-42b2-9d77-96ce8f7075b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:15.271726 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.auth.core [None req-48523629-fa14-42b2-9d77-96ce8f7075b3 None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 11:57:15.273741 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-48523629-fa14-42b2-9d77-96ce8f7075b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 11:57:15.352212 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 79/161] 10.222.0.23 () {58 vars in 1023 bytes} [Thu May 19 11:57:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 11:57:15.353730 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d33971-20bc-473a-b77f-fedc3e3bf9ca None None] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:15.354027 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d33971-20bc-473a-b77f-fedc3e3bf9ca None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:15.354309 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d33971-20bc-473a-b77f-fedc3e3bf9ca None None] PATH_INFO: `/` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:15.354920 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ad48ff-008e-4a2b-8a7d-329db8a362c8 None None] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:57:15.355915 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 80/162] 10.222.0.23 () {54 vars in 926 bytes} [Thu May 19 11:57:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 11:57:15.356651 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d33971-20bc-473a-b77f-fedc3e3bf9ca None None] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:57:15.399479 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9882d7af-ea5b-417f-bc8f-3c89c551cf78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:57:15.400748 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9882d7af-ea5b-417f-bc8f-3c89c551cf78 None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:15.401268 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9882d7af-ea5b-417f-bc8f-3c89c551cf78 None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:15.401740 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9882d7af-ea5b-417f-bc8f-3c89c551cf78 None admin] PATH_INFO: `/v3/users/267c35a521bb473ea2784646137bebda` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:15.403655 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9882d7af-ea5b-417f-bc8f-3c89c551cf78 None admin] RBAC: Authorizing `identity:get_user(user_id=267c35a521bb473ea2784646137bebda)` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:57:15.405717 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9882d7af-ea5b-417f-bc8f-3c89c551cf78 None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:57:15.408917 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 83/163] 10.222.0.23 () {56 vars in 1177 bytes} [Thu May 19 11:57:15 2022] GET /identity/v3/users/267c35a521bb473ea2784646137bebda => generated 275 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:57:15.410585 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd75be49-c977-41fe-bfda-52f3b7641cae None None] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:15.411158 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd75be49-c977-41fe-bfda-52f3b7641cae None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:15.412110 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd75be49-c977-41fe-bfda-52f3b7641cae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:15.415545 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:57:15.419922 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:57:15.436162 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.auth.core [None req-dd75be49-c977-41fe-bfda-52f3b7641cae None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 11:57:15.438686 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-dd75be49-c977-41fe-bfda-52f3b7641cae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 11:57:15.475770 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d33971-20bc-473a-b77f-fedc3e3bf9ca None None] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:57:15.515178 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6216db-6d5d-4d82-98f8-3fe63f2ef82f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:57:15.515910 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6216db-6d5d-4d82-98f8-3fe63f2ef82f None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:15.516734 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6216db-6d5d-4d82-98f8-3fe63f2ef82f None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:15.517351 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6216db-6d5d-4d82-98f8-3fe63f2ef82f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:15.518408 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6216db-6d5d-4d82-98f8-3fe63f2ef82f None admin] RBAC: Adding query filter params (role.id=9db5671c65d3414ea2682afa2ccf1403, scope.project.id=64092131887b4859ae47a1f4c3e0d24a, user.id=11855e565a88475189d279de9469c271) {{(pid=80539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 11:57:15.518976 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6216db-6d5d-4d82-98f8-3fe63f2ef82f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:57:15.520828 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6216db-6d5d-4d82-98f8-3fe63f2ef82f None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:57:15.524341 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 84/164] 10.222.0.23 () {58 vars in 1023 bytes} [Thu May 19 11:57:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 11:57:15.525718 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd75be49-c977-41fe-bfda-52f3b7641cae None None] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:57:15.528441 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 81/165] 10.222.0.23 () {56 vars in 1374 bytes} [Thu May 19 11:57:15 2022] GET /identity/v3/role_assignments?user.id=11855e565a88475189d279de9469c271&scope.project.id=64092131887b4859ae47a1f4c3e0d24a&role.id=9db5671c65d3414ea2682afa2ccf1403 => generated 608 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:57:15.529525 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6216db-6d5d-4d82-98f8-3fe63f2ef82f None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:57:15.706828 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd75be49-c977-41fe-bfda-52f3b7641cae None None] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:57:15.750565 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862b8baa-f031-4685-9d34-d8a2949e8d69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:57:15.751736 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862b8baa-f031-4685-9d34-d8a2949e8d69 None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:15.752191 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862b8baa-f031-4685-9d34-d8a2949e8d69 None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:15.752606 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862b8baa-f031-4685-9d34-d8a2949e8d69 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:15.758297 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:57:15.771188 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862b8baa-f031-4685-9d34-d8a2949e8d69 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 11:57:15.787382 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application [None req-862b8baa-f031-4685-9d34-d8a2949e8d69 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 11:57:15.787382 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 11:57:15.787382 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 11:57:15.787382 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 11:57:15.787382 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 11:57:15.787382 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 11:57:15.787382 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 11:57:15.787382 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 11:57:15.787382 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 11:57:15.787382 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 11:57:15.787382 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 11:57:15.787382 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 11:57:15.787382 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 11:57:15.787382 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 11:57:15.787382 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 11:57:15.787382 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 11:57:15.788541 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 11:57:15.788541 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 11:57:15.788541 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 11:57:15.788541 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 11:57:15.788541 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 11:57:15.788541 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 11:57:15.788541 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 11:57:15.788541 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 11:57:15.788541 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 11:57:15.788541 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.get_or_create( May 19 11:57:15.788541 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 11:57:15.788541 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application with Lock( May 19 11:57:15.788541 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 11:57:15.788541 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._enter() May 19 11:57:15.788541 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 11:57:15.788541 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 11:57:15.789496 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 11:57:15.789496 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.creator() May 19 11:57:15.789496 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 11:57:15.789496 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application created_value = creator( May 19 11:57:15.789496 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 11:57:15.789496 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 11:57:15.789496 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 11:57:15.789496 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 11:57:15.789496 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 11:57:15.789496 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 11:57:15.789496 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 11:57:15.789496 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application May 19 11:57:15.792008 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 85/166] 10.222.0.23 () {56 vars in 1114 bytes} [Thu May 19 11:57:15 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 11:57:15.792986 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862b8baa-f031-4685-9d34-d8a2949e8d69 None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:57:15.803875 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6216db-6d5d-4d82-98f8-3fe63f2ef82f None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:57:15.832376 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:57:15.878013 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57564691-7914-4d1e-9385-b1dc2574f1cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:57:15.878013 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57564691-7914-4d1e-9385-b1dc2574f1cc None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:15.878335 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57564691-7914-4d1e-9385-b1dc2574f1cc None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:15.878430 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57564691-7914-4d1e-9385-b1dc2574f1cc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:15.919659 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862b8baa-f031-4685-9d34-d8a2949e8d69 None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:57:15.983891 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57564691-7914-4d1e-9385-b1dc2574f1cc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:57:15.984388 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57564691-7914-4d1e-9385-b1dc2574f1cc None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:57:15.989734 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83be2ae6-5600-489c-a83b-eebd15abd071 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:57:15.990920 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83be2ae6-5600-489c-a83b-eebd15abd071 None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:15.991352 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83be2ae6-5600-489c-a83b-eebd15abd071 None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:15.991766 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83be2ae6-5600-489c-a83b-eebd15abd071 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:16.006650 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83be2ae6-5600-489c-a83b-eebd15abd071 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 11:57:16.015026 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application [None req-83be2ae6-5600-489c-a83b-eebd15abd071 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 11:57:16.015026 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 11:57:16.015026 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 11:57:16.015026 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 11:57:16.015026 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 11:57:16.015026 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 11:57:16.015026 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 11:57:16.015026 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 11:57:16.015026 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 11:57:16.015026 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 11:57:16.015026 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 11:57:16.015026 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 11:57:16.015026 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 11:57:16.015026 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 11:57:16.015026 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 11:57:16.015026 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 11:57:16.017132 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 11:57:16.017132 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 11:57:16.017132 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 11:57:16.017132 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 11:57:16.017132 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 11:57:16.017132 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 11:57:16.017132 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 11:57:16.017132 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 11:57:16.017132 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 11:57:16.017132 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.get_or_create( May 19 11:57:16.017132 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 11:57:16.017132 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application with Lock( May 19 11:57:16.017132 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 11:57:16.017132 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._enter() May 19 11:57:16.017132 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 11:57:16.017132 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 11:57:16.018116 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 11:57:16.018116 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.creator() May 19 11:57:16.018116 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 11:57:16.018116 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application created_value = creator( May 19 11:57:16.018116 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 11:57:16.018116 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 11:57:16.018116 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 11:57:16.018116 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 11:57:16.018116 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 11:57:16.018116 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 11:57:16.018116 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 11:57:16.018116 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application May 19 11:57:16.020320 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 86/167] 10.222.0.23 () {56 vars in 1099 bytes} [Thu May 19 11:57:15 2022] GET /identity/v3/roles/member => generated 84 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 11:57:16.021362 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83be2ae6-5600-489c-a83b-eebd15abd071 None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:57:16.095012 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:57:16.143533 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83be2ae6-5600-489c-a83b-eebd15abd071 None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:57:16.181873 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a73ba3-8dc1-4223-ab35-0633ab7f572b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:57:16.182624 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 82/168] 10.222.0.23 () {58 vars in 1282 bytes} [Thu May 19 11:57:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 653 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 11:57:16.182943 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a73ba3-8dc1-4223-ab35-0633ab7f572b None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:16.183368 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a73ba3-8dc1-4223-ab35-0633ab7f572b None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:16.183573 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57564691-7914-4d1e-9385-b1dc2574f1cc None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:57:16.183978 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a73ba3-8dc1-4223-ab35-0633ab7f572b None admin] PATH_INFO: `/v3/roles` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:16.184919 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a73ba3-8dc1-4223-ab35-0633ab7f572b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 11:57:16.185416 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a73ba3-8dc1-4223-ab35-0633ab7f572b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:57:16.187899 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a73ba3-8dc1-4223-ab35-0633ab7f572b None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:57:16.197693 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 87/169] 10.222.0.23 () {56 vars in 1111 bytes} [Thu May 19 11:57:16 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:57:16.198774 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a73ba3-8dc1-4223-ab35-0633ab7f572b None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:57:16.209510 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:57:16.309353 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a73ba3-8dc1-4223-ab35-0633ab7f572b None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:57:16.327442 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:57:16.347431 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f0a4d1-0fd2-422a-a311-23e9ffc0e616 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:57:16.348769 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f0a4d1-0fd2-422a-a311-23e9ffc0e616 None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:16.349225 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f0a4d1-0fd2-422a-a311-23e9ffc0e616 None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:16.349702 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f0a4d1-0fd2-422a-a311-23e9ffc0e616 None admin] PATH_INFO: `/v3/projects/64092131887b4859ae47a1f4c3e0d24a` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:16.351111 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f0a4d1-0fd2-422a-a311-23e9ffc0e616 None admin] RBAC: Authorizing `identity:get_project(project_id=64092131887b4859ae47a1f4c3e0d24a)` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:57:16.352831 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f0a4d1-0fd2-422a-a311-23e9ffc0e616 None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:57:16.355153 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 88/170] 10.222.0.23 () {56 vars in 1186 bytes} [Thu May 19 11:57:16 2022] GET /identity/v3/projects/64092131887b4859ae47a1f4c3e0d24a => generated 298 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:57:16.356252 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f0a4d1-0fd2-422a-a311-23e9ffc0e616 None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:57:16.371347 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57564691-7914-4d1e-9385-b1dc2574f1cc None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:57:16.433479 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f41572b-212b-47ad-bb9e-371325be1bf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:57:16.434280 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f41572b-212b-47ad-bb9e-371325be1bf6 None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:16.434655 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f41572b-212b-47ad-bb9e-371325be1bf6 None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:16.435012 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f41572b-212b-47ad-bb9e-371325be1bf6 None admin] PATH_INFO: `/v3/roles` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:16.435878 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f41572b-212b-47ad-bb9e-371325be1bf6 None admin] RBAC: Adding query filter params (name=member) {{(pid=80539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 11:57:16.436259 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f41572b-212b-47ad-bb9e-371325be1bf6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:57:16.438372 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f41572b-212b-47ad-bb9e-371325be1bf6 None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:57:16.455750 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 83/171] 10.222.0.23 () {56 vars in 1101 bytes} [Thu May 19 11:57:16 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:57:16.456971 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f41572b-212b-47ad-bb9e-371325be1bf6 None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:57:16.470635 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f0a4d1-0fd2-422a-a311-23e9ffc0e616 None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:57:16.545133 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6476a053-7660-4660-87bf-5ed2b56b0dbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:57:16.546066 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6476a053-7660-4660-87bf-5ed2b56b0dbd None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:16.546362 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6476a053-7660-4660-87bf-5ed2b56b0dbd None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:16.547266 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6476a053-7660-4660-87bf-5ed2b56b0dbd None admin] PATH_INFO: `/v3/users/267c35a521bb473ea2784646137bebda` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:16.552667 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6476a053-7660-4660-87bf-5ed2b56b0dbd None admin] RBAC: Authorizing `identity:get_user(user_id=267c35a521bb473ea2784646137bebda)` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:57:16.554273 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6476a053-7660-4660-87bf-5ed2b56b0dbd None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:57:16.556504 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 89/172] 10.222.0.23 () {56 vars in 1177 bytes} [Thu May 19 11:57:16 2022] GET /identity/v3/users/267c35a521bb473ea2784646137bebda => generated 275 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:57:16.558684 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6476a053-7660-4660-87bf-5ed2b56b0dbd None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:57:16.627635 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f41572b-212b-47ad-bb9e-371325be1bf6 None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:57:16.632108 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:57:16.687938 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ececcb4-7c2d-400c-a2cf-86bf55b6b340 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:57:16.688723 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ececcb4-7c2d-400c-a2cf-86bf55b6b340 None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:16.689525 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ececcb4-7c2d-400c-a2cf-86bf55b6b340 None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:16.689861 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ececcb4-7c2d-400c-a2cf-86bf55b6b340 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:16.690764 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ececcb4-7c2d-400c-a2cf-86bf55b6b340 None admin] RBAC: Adding query filter params (role.id=259eb400cdb4439db72d14456ae9cd08, scope.project.id=64092131887b4859ae47a1f4c3e0d24a, user.id=267c35a521bb473ea2784646137bebda) {{(pid=80539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 11:57:16.691090 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ececcb4-7c2d-400c-a2cf-86bf55b6b340 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:57:16.692671 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ececcb4-7c2d-400c-a2cf-86bf55b6b340 None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:57:16.701723 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:57:16.713062 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 84/173] 10.222.0.23 () {56 vars in 1374 bytes} [Thu May 19 11:57:16 2022] GET /identity/v3/role_assignments?user.id=267c35a521bb473ea2784646137bebda&scope.project.id=64092131887b4859ae47a1f4c3e0d24a&role.id=259eb400cdb4439db72d14456ae9cd08 => generated 608 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:57:16.714087 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ececcb4-7c2d-400c-a2cf-86bf55b6b340 None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:57:16.902183 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6476a053-7660-4660-87bf-5ed2b56b0dbd None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:57:16.939727 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ececcb4-7c2d-400c-a2cf-86bf55b6b340 None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:57:17.019388 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e60100-250a-46c4-863d-f967b7d2c9e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:57:17.020096 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e60100-250a-46c4-863d-f967b7d2c9e6 None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:17.020388 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e60100-250a-46c4-863d-f967b7d2c9e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:17.020786 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e60100-250a-46c4-863d-f967b7d2c9e6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:17.036529 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc76fecf-7cdb-4c16-a9f9-099e14beb512 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:57:17.037508 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc76fecf-7cdb-4c16-a9f9-099e14beb512 None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:17.037797 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc76fecf-7cdb-4c16-a9f9-099e14beb512 None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:17.038079 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc76fecf-7cdb-4c16-a9f9-099e14beb512 None admin] PATH_INFO: `/v3/users/267c35a521bb473ea2784646137bebda` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:17.047348 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc76fecf-7cdb-4c16-a9f9-099e14beb512 None admin] RBAC: Authorizing `identity:get_user(user_id=267c35a521bb473ea2784646137bebda)` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:57:17.047348 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc76fecf-7cdb-4c16-a9f9-099e14beb512 None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:57:17.047348 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 90/174] 10.222.0.23 () {56 vars in 1177 bytes} [Thu May 19 11:57:16 2022] GET /identity/v3/users/267c35a521bb473ea2784646137bebda => generated 275 bytes in 490 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:57:17.048735 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce48b50-3898-455d-a427-ca641f57650a None None] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:17.049391 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:57:17.050130 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce48b50-3898-455d-a427-ca641f57650a None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:17.050130 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce48b50-3898-455d-a427-ca641f57650a None None] PATH_INFO: `/` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:17.052191 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 91/175] 10.222.0.23 () {54 vars in 926 bytes} [Thu May 19 11:57:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 11:57:17.062289 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242d2366-7a0d-47c5-b95d-36cb569f671c None None] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:17.062766 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242d2366-7a0d-47c5-b95d-36cb569f671c None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:17.063236 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242d2366-7a0d-47c5-b95d-36cb569f671c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:17.109071 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.auth.core [None req-242d2366-7a0d-47c5-b95d-36cb569f671c None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 11:57:17.110518 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-242d2366-7a0d-47c5-b95d-36cb569f671c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 11:57:17.209768 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e60100-250a-46c4-863d-f967b7d2c9e6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:57:17.211580 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e60100-250a-46c4-863d-f967b7d2c9e6 None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:57:17.293200 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 92/176] 10.222.0.23 () {58 vars in 1023 bytes} [Thu May 19 11:57:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 11:57:17.294407 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242d2366-7a0d-47c5-b95d-36cb569f671c None None] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:57:17.417395 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:57:17.528818 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 85/177] 10.222.0.23 () {58 vars in 1282 bytes} [Thu May 19 11:57:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 815 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 11:57:17.529626 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e60100-250a-46c4-863d-f967b7d2c9e6 None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:57:17.551364 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242d2366-7a0d-47c5-b95d-36cb569f671c None None] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:57:17.602742 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:57:17.603601 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b4eae8-8baf-47c0-80b5-f862c0645233 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:57:17.604297 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b4eae8-8baf-47c0-80b5-f862c0645233 None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:17.604589 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b4eae8-8baf-47c0-80b5-f862c0645233 None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:17.604888 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b4eae8-8baf-47c0-80b5-f862c0645233 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:17.702454 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e60100-250a-46c4-863d-f967b7d2c9e6 None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:57:17.707613 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b4eae8-8baf-47c0-80b5-f862c0645233 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:57:17.709545 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b4eae8-8baf-47c0-80b5-f862c0645233 None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:57:17.740618 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515fd740-3eb4-4b96-883a-0bccd89f4afc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:57:17.741335 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515fd740-3eb4-4b96-883a-0bccd89f4afc None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:17.741659 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515fd740-3eb4-4b96-883a-0bccd89f4afc None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:17.741945 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515fd740-3eb4-4b96-883a-0bccd89f4afc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:17.753056 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515fd740-3eb4-4b96-883a-0bccd89f4afc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 11:57:17.761224 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application [None req-515fd740-3eb4-4b96-883a-0bccd89f4afc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 11:57:17.761224 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 11:57:17.761224 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 11:57:17.761224 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 11:57:17.761224 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 11:57:17.761224 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 11:57:17.761224 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 11:57:17.761224 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 11:57:17.761224 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 11:57:17.761224 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 11:57:17.761224 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 11:57:17.761224 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 11:57:17.761224 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 11:57:17.761224 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 11:57:17.761224 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 11:57:17.761224 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 11:57:17.762315 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 11:57:17.762315 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 11:57:17.762315 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 11:57:17.762315 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 11:57:17.762315 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 11:57:17.762315 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 11:57:17.762315 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 11:57:17.762315 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 11:57:17.762315 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 11:57:17.762315 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.get_or_create( May 19 11:57:17.762315 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 11:57:17.762315 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application with Lock( May 19 11:57:17.762315 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 11:57:17.762315 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._enter() May 19 11:57:17.762315 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 11:57:17.762315 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 11:57:17.763258 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 11:57:17.763258 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.creator() May 19 11:57:17.763258 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 11:57:17.763258 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application created_value = creator( May 19 11:57:17.763258 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 11:57:17.763258 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 11:57:17.763258 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 11:57:17.763258 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 11:57:17.763258 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 11:57:17.763258 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 11:57:17.763258 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 11:57:17.763258 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application May 19 11:57:17.765783 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 86/178] 10.222.0.23 () {56 vars in 1096 bytes} [Thu May 19 11:57:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 11:57:17.766580 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515fd740-3eb4-4b96-883a-0bccd89f4afc None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:57:17.774134 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:57:17.836322 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 93/179] 10.222.0.23 () {58 vars in 1282 bytes} [Thu May 19 11:57:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 542 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 11:57:17.841404 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db0db97-3a44-4b04-9ee1-45a572093929 None None] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:17.842032 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db0db97-3a44-4b04-9ee1-45a572093929 None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:17.842032 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db0db97-3a44-4b04-9ee1-45a572093929 None None] PATH_INFO: `/` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:17.844244 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 94/180] 10.222.0.23 () {54 vars in 926 bytes} [Thu May 19 11:57:17 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 11:57:17.850162 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c90a072-6bab-41c6-8281-19427becbbec None None] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:17.850454 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c90a072-6bab-41c6-8281-19427becbbec None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:17.850742 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c90a072-6bab-41c6-8281-19427becbbec None None] PATH_INFO: `/` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:17.852119 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 95/181] 10.222.0.23 () {54 vars in 926 bytes} [Thu May 19 11:57:17 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 11:57:17.856688 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c90a072-6bab-41c6-8281-19427becbbec None None] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:57:17.893044 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:57:17.897676 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515fd740-3eb4-4b96-883a-0bccd89f4afc None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:57:17.947903 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408b0291-88f3-4658-9c8f-eb1ba74f3ef2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:57:17.948791 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408b0291-88f3-4658-9c8f-eb1ba74f3ef2 None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:17.949076 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408b0291-88f3-4658-9c8f-eb1ba74f3ef2 None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:17.949356 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408b0291-88f3-4658-9c8f-eb1ba74f3ef2 None admin] PATH_INFO: `/v3/projects/64092131887b4859ae47a1f4c3e0d24a` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:17.950583 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408b0291-88f3-4658-9c8f-eb1ba74f3ef2 None admin] RBAC: Authorizing `identity:get_project(project_id=64092131887b4859ae47a1f4c3e0d24a)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:57:17.952098 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408b0291-88f3-4658-9c8f-eb1ba74f3ef2 None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:57:17.954366 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 87/182] 10.222.0.23 () {56 vars in 1186 bytes} [Thu May 19 11:57:17 2022] GET /identity/v3/projects/64092131887b4859ae47a1f4c3e0d24a => generated 298 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:57:17.954812 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408b0291-88f3-4658-9c8f-eb1ba74f3ef2 None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:57:17.992988 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c90a072-6bab-41c6-8281-19427becbbec None None] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:57:18.028286 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f5a578-b343-465b-998f-5ac13b41513b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:57:18.029143 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f5a578-b343-465b-998f-5ac13b41513b None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:18.029348 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f5a578-b343-465b-998f-5ac13b41513b None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:18.029616 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f5a578-b343-465b-998f-5ac13b41513b None admin] PATH_INFO: `/v3/roles` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:18.030354 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f5a578-b343-465b-998f-5ac13b41513b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 11:57:18.033555 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f5a578-b343-465b-998f-5ac13b41513b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:57:18.033555 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f5a578-b343-465b-998f-5ac13b41513b None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:57:18.041631 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 96/183] 10.222.0.23 () {56 vars in 1099 bytes} [Thu May 19 11:57:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:57:18.043072 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c61ccb-fe6d-40ab-b988-cf7823c93b77 None None] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:18.043404 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c61ccb-fe6d-40ab-b988-cf7823c93b77 None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:18.043735 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c61ccb-fe6d-40ab-b988-cf7823c93b77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:18.060924 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.auth.core [None req-25c61ccb-fe6d-40ab-b988-cf7823c93b77 None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 11:57:18.062128 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-25c61ccb-fe6d-40ab-b988-cf7823c93b77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 11:57:18.066227 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408b0291-88f3-4658-9c8f-eb1ba74f3ef2 None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:57:18.070311 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:57:18.100317 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66280dce-c9c1-476e-b05a-e7aeca930cf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:57:18.101078 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66280dce-c9c1-476e-b05a-e7aeca930cf7 None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:18.101364 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66280dce-c9c1-476e-b05a-e7aeca930cf7 None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:18.101649 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66280dce-c9c1-476e-b05a-e7aeca930cf7 None admin] PATH_INFO: `/v3/projects/4aeddbfe954d472aa273e1df1efca1fd` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:18.102829 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66280dce-c9c1-476e-b05a-e7aeca930cf7 None admin] RBAC: Authorizing `identity:get_project(project_id=4aeddbfe954d472aa273e1df1efca1fd)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:57:18.104319 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66280dce-c9c1-476e-b05a-e7aeca930cf7 None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:57:18.106022 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 88/184] 10.222.0.23 () {56 vars in 1186 bytes} [Thu May 19 11:57:17 2022] GET /identity/v3/projects/4aeddbfe954d472aa273e1df1efca1fd => generated 312 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:57:18.107189 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179542b6-b15b-491a-a9d6-4c1d81338488 None None] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:18.107485 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179542b6-b15b-491a-a9d6-4c1d81338488 None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:18.107750 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179542b6-b15b-491a-a9d6-4c1d81338488 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:18.125724 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.auth.core [None req-179542b6-b15b-491a-a9d6-4c1d81338488 None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 11:57:18.126946 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-179542b6-b15b-491a-a9d6-4c1d81338488 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 11:57:18.139342 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 97/185] 10.222.0.23 () {58 vars in 1023 bytes} [Thu May 19 11:57:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 11:57:18.140562 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c61ccb-fe6d-40ab-b988-cf7823c93b77 None None] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:57:18.166593 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:57:18.223968 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 89/186] 10.222.0.23 () {58 vars in 1023 bytes} [Thu May 19 11:57:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 11:57:18.225929 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef11e099-633b-44eb-a5d2-9adc370f48d2 None None] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:18.226218 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef11e099-633b-44eb-a5d2-9adc370f48d2 None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:18.226501 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef11e099-633b-44eb-a5d2-9adc370f48d2 None None] PATH_INFO: `/` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:18.228922 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 90/187] 10.222.0.23 () {54 vars in 926 bytes} [Thu May 19 11:57:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 11:57:18.228922 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef11e099-633b-44eb-a5d2-9adc370f48d2 None None] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:57:18.248749 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c61ccb-fe6d-40ab-b988-cf7823c93b77 None None] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:57:18.288014 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b814bc6-b898-46b1-a1eb-3582b80f77e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:57:18.288676 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b814bc6-b898-46b1-a1eb-3582b80f77e1 None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:18.288964 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b814bc6-b898-46b1-a1eb-3582b80f77e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:18.289243 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b814bc6-b898-46b1-a1eb-3582b80f77e1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:18.291863 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b814bc6-b898-46b1-a1eb-3582b80f77e1 None admin] RBAC: Adding query filter params (role.id=8215583bb4224c75b093ae8e3993eb9f, scope.project.id=64092131887b4859ae47a1f4c3e0d24a, user.id=267c35a521bb473ea2784646137bebda) {{(pid=80540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 11:57:18.291863 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b814bc6-b898-46b1-a1eb-3582b80f77e1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:57:18.291863 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b814bc6-b898-46b1-a1eb-3582b80f77e1 None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:57:18.298431 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 98/188] 10.222.0.23 () {56 vars in 1374 bytes} [Thu May 19 11:57:18 2022] GET /identity/v3/role_assignments?user.id=267c35a521bb473ea2784646137bebda&scope.project.id=64092131887b4859ae47a1f4c3e0d24a&role.id=8215583bb4224c75b093ae8e3993eb9f => generated 608 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:57:18.299226 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b814bc6-b898-46b1-a1eb-3582b80f77e1 None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:57:18.308132 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:57:18.369380 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef11e099-633b-44eb-a5d2-9adc370f48d2 None None] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:57:18.415503 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5342d49-f4ca-4066-8fb2-61c8757aa144 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:57:18.416896 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5342d49-f4ca-4066-8fb2-61c8757aa144 None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:18.417327 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5342d49-f4ca-4066-8fb2-61c8757aa144 None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:18.417743 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5342d49-f4ca-4066-8fb2-61c8757aa144 None admin] PATH_INFO: `/v3/users/4105c099a7824283aa03a6c3cc324850` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:18.444378 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b814bc6-b898-46b1-a1eb-3582b80f77e1 None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:57:18.445952 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5342d49-f4ca-4066-8fb2-61c8757aa144 None admin] RBAC: Authorizing `identity:get_user(user_id=4105c099a7824283aa03a6c3cc324850)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:57:18.449106 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5342d49-f4ca-4066-8fb2-61c8757aa144 None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:57:18.452418 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 91/189] 10.222.0.23 () {56 vars in 1177 bytes} [Thu May 19 11:57:18 2022] GET /identity/v3/users/4105c099a7824283aa03a6c3cc324850 => generated 283 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:57:18.455474 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5342d49-f4ca-4066-8fb2-61c8757aa144 None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:57:18.486267 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:57:18.487942 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ce6ae2-c1d2-4f5b-8eb9-a53cb2fd3991 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:57:18.488625 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ce6ae2-c1d2-4f5b-8eb9-a53cb2fd3991 None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:18.488910 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ce6ae2-c1d2-4f5b-8eb9-a53cb2fd3991 None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:18.489301 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ce6ae2-c1d2-4f5b-8eb9-a53cb2fd3991 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:18.490141 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ce6ae2-c1d2-4f5b-8eb9-a53cb2fd3991 None admin] RBAC: Adding query filter params (role.id=259eb400cdb4439db72d14456ae9cd08, scope.project.id=4aeddbfe954d472aa273e1df1efca1fd, user.id=267c35a521bb473ea2784646137bebda) {{(pid=80540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 11:57:18.490455 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ce6ae2-c1d2-4f5b-8eb9-a53cb2fd3991 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:57:18.491988 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ce6ae2-c1d2-4f5b-8eb9-a53cb2fd3991 None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:57:18.498700 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 99/190] 10.222.0.23 () {56 vars in 1374 bytes} [Thu May 19 11:57:18 2022] GET /identity/v3/role_assignments?user.id=267c35a521bb473ea2784646137bebda&scope.project.id=4aeddbfe954d472aa273e1df1efca1fd&role.id=259eb400cdb4439db72d14456ae9cd08 => generated 608 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:57:18.499571 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ce6ae2-c1d2-4f5b-8eb9-a53cb2fd3991 None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:57:18.593519 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5342d49-f4ca-4066-8fb2-61c8757aa144 None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:57:18.612647 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:57:18.655221 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e29a56a-de1f-4f5c-a818-3373b5b89a3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:57:18.659488 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ce6ae2-c1d2-4f5b-8eb9-a53cb2fd3991 None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:57:18.659644 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e29a56a-de1f-4f5c-a818-3373b5b89a3b None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:18.659644 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e29a56a-de1f-4f5c-a818-3373b5b89a3b None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:18.659644 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e29a56a-de1f-4f5c-a818-3373b5b89a3b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:18.671048 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e29a56a-de1f-4f5c-a818-3373b5b89a3b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 11:57:18.680308 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application [None req-2e29a56a-de1f-4f5c-a818-3373b5b89a3b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 11:57:18.680308 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 11:57:18.680308 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 11:57:18.680308 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 11:57:18.680308 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 11:57:18.680308 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 11:57:18.680308 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 11:57:18.680308 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 11:57:18.680308 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 11:57:18.680308 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 11:57:18.680308 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 11:57:18.680308 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 11:57:18.680308 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 11:57:18.680308 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 11:57:18.680308 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 11:57:18.680308 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 11:57:18.683177 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 11:57:18.683177 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 11:57:18.683177 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 11:57:18.683177 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 11:57:18.683177 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 11:57:18.683177 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 11:57:18.683177 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 11:57:18.683177 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 11:57:18.683177 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 11:57:18.683177 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.get_or_create( May 19 11:57:18.683177 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 11:57:18.683177 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application with Lock( May 19 11:57:18.683177 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 11:57:18.683177 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._enter() May 19 11:57:18.683177 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 11:57:18.683177 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 11:57:18.686127 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 11:57:18.686127 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.creator() May 19 11:57:18.686127 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 11:57:18.686127 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application created_value = creator( May 19 11:57:18.686127 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 11:57:18.686127 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 11:57:18.686127 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 11:57:18.686127 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 11:57:18.686127 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 11:57:18.686127 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 11:57:18.686127 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 11:57:18.686127 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application May 19 11:57:18.686127 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 92/191] 10.222.0.23 () {56 vars in 1114 bytes} [Thu May 19 11:57:18 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 11:57:18.686127 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ef0f83-237c-41f1-8123-bdbbac6c2e3e None None] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:18.687051 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ef0f83-237c-41f1-8123-bdbbac6c2e3e None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:18.687051 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ef0f83-237c-41f1-8123-bdbbac6c2e3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:18.707200 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.auth.core [None req-52ef0f83-237c-41f1-8123-bdbbac6c2e3e None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 11:57:18.708482 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-52ef0f83-237c-41f1-8123-bdbbac6c2e3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 11:57:18.726479 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164ad466-c77b-4acd-a2be-caf13d2443a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:57:18.727474 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164ad466-c77b-4acd-a2be-caf13d2443a5 None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:18.727767 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164ad466-c77b-4acd-a2be-caf13d2443a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:18.728424 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164ad466-c77b-4acd-a2be-caf13d2443a5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:18.729625 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164ad466-c77b-4acd-a2be-caf13d2443a5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:57:18.731848 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164ad466-c77b-4acd-a2be-caf13d2443a5 None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:57:18.733283 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 100/192] 10.222.0.23 () {56 vars in 1108 bytes} [Thu May 19 11:57:18 2022] GET /identity/v3/domains/default => generated 199 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:57:18.735405 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164ad466-c77b-4acd-a2be-caf13d2443a5 None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:57:18.791855 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 93/193] 10.222.0.23 () {58 vars in 1023 bytes} [Thu May 19 11:57:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 11:57:18.792722 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ef0f83-237c-41f1-8123-bdbbac6c2e3e None None] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:57:18.808010 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:57:18.850354 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164ad466-c77b-4acd-a2be-caf13d2443a5 None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:57:18.886164 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef05da9-d0b5-422c-b5d5-aceb1d41306d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:57:18.886822 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef05da9-d0b5-422c-b5d5-aceb1d41306d None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:18.887111 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef05da9-d0b5-422c-b5d5-aceb1d41306d None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:18.887399 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef05da9-d0b5-422c-b5d5-aceb1d41306d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:18.895676 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ef0f83-237c-41f1-8123-bdbbac6c2e3e None None] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:57:18.897794 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:57:18.935707 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbdeb260-8d3c-4908-9d5e-cad5f06d4ef1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:57:18.936338 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdeb260-8d3c-4908-9d5e-cad5f06d4ef1 None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:18.936627 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdeb260-8d3c-4908-9d5e-cad5f06d4ef1 None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:18.936904 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdeb260-8d3c-4908-9d5e-cad5f06d4ef1 None admin] PATH_INFO: `/v3/roles` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:18.937818 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbdeb260-8d3c-4908-9d5e-cad5f06d4ef1 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 11:57:18.938124 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbdeb260-8d3c-4908-9d5e-cad5f06d4ef1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:57:18.939548 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbdeb260-8d3c-4908-9d5e-cad5f06d4ef1 None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:57:18.949079 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 94/194] 10.222.0.23 () {56 vars in 1111 bytes} [Thu May 19 11:57:18 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:57:18.949792 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbdeb260-8d3c-4908-9d5e-cad5f06d4ef1 None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:57:18.975356 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef05da9-d0b5-422c-b5d5-aceb1d41306d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:57:18.976944 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef05da9-d0b5-422c-b5d5-aceb1d41306d None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:57:19.036279 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:57:19.065347 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbdeb260-8d3c-4908-9d5e-cad5f06d4ef1 None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:57:19.089685 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 101/195] 10.222.0.23 () {58 vars in 1282 bytes} [Thu May 19 11:57:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 11:57:19.090552 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef05da9-d0b5-422c-b5d5-aceb1d41306d None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:57:19.114769 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91cb43bc-8ae4-4933-87d4-14b58982b56c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:57:19.115447 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cb43bc-8ae4-4933-87d4-14b58982b56c None admin] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:19.115731 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cb43bc-8ae4-4933-87d4-14b58982b56c None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:19.116031 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cb43bc-8ae4-4933-87d4-14b58982b56c None admin] PATH_INFO: `/v3/users` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:19.116744 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91cb43bc-8ae4-4933-87d4-14b58982b56c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:57:19.118205 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91cb43bc-8ae4-4933-87d4-14b58982b56c None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:57:19.136200 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:57:19.196064 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef05da9-d0b5-422c-b5d5-aceb1d41306d None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:57:19.217073 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 95/196] 10.222.0.23 () {60 vars in 1132 bytes} [Thu May 19 11:57:18 2022] POST /identity/v3/users => generated 297 bytes in 268 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 11:57:19.217925 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91cb43bc-8ae4-4933-87d4-14b58982b56c None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:57:19.246361 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d3ffa0-8a1d-494b-8989-be19ce71b886 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:57:19.248334 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d3ffa0-8a1d-494b-8989-be19ce71b886 None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:19.248817 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d3ffa0-8a1d-494b-8989-be19ce71b886 None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:19.249107 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d3ffa0-8a1d-494b-8989-be19ce71b886 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:19.264732 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d3ffa0-8a1d-494b-8989-be19ce71b886 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 11:57:19.273798 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application [None req-28d3ffa0-8a1d-494b-8989-be19ce71b886 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 19 11:57:19.273798 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 11:57:19.273798 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 11:57:19.273798 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 11:57:19.273798 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 11:57:19.273798 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 11:57:19.273798 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 11:57:19.273798 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 11:57:19.273798 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 11:57:19.273798 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 11:57:19.273798 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 11:57:19.273798 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 11:57:19.273798 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 11:57:19.273798 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 11:57:19.273798 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 11:57:19.273798 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 11:57:19.274794 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 11:57:19.274794 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 11:57:19.274794 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 11:57:19.274794 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 11:57:19.274794 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 11:57:19.274794 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 11:57:19.274794 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 11:57:19.274794 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 11:57:19.274794 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 11:57:19.274794 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.get_or_create( May 19 11:57:19.274794 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 11:57:19.274794 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application with Lock( May 19 11:57:19.274794 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 11:57:19.274794 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._enter() May 19 11:57:19.274794 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 11:57:19.274794 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 11:57:19.275581 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 11:57:19.275581 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.creator() May 19 11:57:19.275581 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 11:57:19.275581 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application created_value = creator( May 19 11:57:19.275581 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 11:57:19.275581 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 11:57:19.275581 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 11:57:19.275581 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 11:57:19.275581 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 11:57:19.275581 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 11:57:19.275581 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 19 11:57:19.275581 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application May 19 11:57:19.275581 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 102/197] 10.222.0.23 () {56 vars in 1099 bytes} [Thu May 19 11:57:19 2022] GET /identity/v3/roles/reader => generated 84 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 11:57:19.277156 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d3ffa0-8a1d-494b-8989-be19ce71b886 None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:57:19.347455 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91cb43bc-8ae4-4933-87d4-14b58982b56c None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:57:19.382128 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:57:19.387233 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d88595-5633-4b01-8552-f77d52a7af0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:57:19.388167 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d88595-5633-4b01-8552-f77d52a7af0c None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:19.388461 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d88595-5633-4b01-8552-f77d52a7af0c None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:19.388737 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d88595-5633-4b01-8552-f77d52a7af0c None admin] PATH_INFO: `/v3/users/4105c099a7824283aa03a6c3cc324850` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:19.390092 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d88595-5633-4b01-8552-f77d52a7af0c None admin] RBAC: Authorizing `identity:get_user(user_id=4105c099a7824283aa03a6c3cc324850)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:57:19.392104 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d88595-5633-4b01-8552-f77d52a7af0c None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:57:19.395397 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 96/198] 10.222.0.23 () {56 vars in 1177 bytes} [Thu May 19 11:57:19 2022] GET /identity/v3/users/4105c099a7824283aa03a6c3cc324850 => generated 283 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:57:19.395397 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d88595-5633-4b01-8552-f77d52a7af0c None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:57:19.402721 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:57:19.407335 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d3ffa0-8a1d-494b-8989-be19ce71b886 None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:57:19.450597 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd07a17-c560-41cc-980e-39ce644e5271 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:57:19.451411 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd07a17-c560-41cc-980e-39ce644e5271 None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:19.451695 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd07a17-c560-41cc-980e-39ce644e5271 None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:19.452000 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd07a17-c560-41cc-980e-39ce644e5271 None admin] PATH_INFO: `/v3/projects/0c5d68529f48460f9f6e03e3f8fb205b` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:19.453550 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd07a17-c560-41cc-980e-39ce644e5271 None admin] RBAC: Authorizing `identity:get_project(project_id=0c5d68529f48460f9f6e03e3f8fb205b)` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:57:19.455442 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd07a17-c560-41cc-980e-39ce644e5271 None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:57:19.457116 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 103/199] 10.222.0.23 () {56 vars in 1186 bytes} [Thu May 19 11:57:19 2022] GET /identity/v3/projects/0c5d68529f48460f9f6e03e3f8fb205b => generated 302 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:57:19.457994 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd07a17-c560-41cc-980e-39ce644e5271 None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:57:19.540093 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d88595-5633-4b01-8552-f77d52a7af0c None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:57:19.587704 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdae9f09-1b2d-4c73-8544-d80a9055e451 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:57:19.588885 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdae9f09-1b2d-4c73-8544-d80a9055e451 None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:19.589387 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdae9f09-1b2d-4c73-8544-d80a9055e451 None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:19.589851 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdae9f09-1b2d-4c73-8544-d80a9055e451 None admin] PATH_INFO: `/v3/roles` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:19.591548 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdae9f09-1b2d-4c73-8544-d80a9055e451 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 11:57:19.592343 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdae9f09-1b2d-4c73-8544-d80a9055e451 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:57:19.594555 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdae9f09-1b2d-4c73-8544-d80a9055e451 None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:57:19.596545 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd07a17-c560-41cc-980e-39ce644e5271 None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:57:19.606309 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 97/200] 10.222.0.23 () {56 vars in 1101 bytes} [Thu May 19 11:57:19 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:57:19.607022 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdae9f09-1b2d-4c73-8544-d80a9055e451 None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:57:19.645680 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e99978-90ef-4cb1-b9d9-06704250a33c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:57:19.646788 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e99978-90ef-4cb1-b9d9-06704250a33c None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:19.647082 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e99978-90ef-4cb1-b9d9-06704250a33c None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:19.647623 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e99978-90ef-4cb1-b9d9-06704250a33c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:19.697737 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:57:19.706534 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:57:19.752606 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e99978-90ef-4cb1-b9d9-06704250a33c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:57:19.754359 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e99978-90ef-4cb1-b9d9-06704250a33c None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:57:19.757673 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdae9f09-1b2d-4c73-8544-d80a9055e451 None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:57:19.846385 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9d5257-3c48-4448-b75e-43c1c7e9d07d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:57:19.847995 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9d5257-3c48-4448-b75e-43c1c7e9d07d None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:19.848741 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9d5257-3c48-4448-b75e-43c1c7e9d07d None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:19.849427 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9d5257-3c48-4448-b75e-43c1c7e9d07d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:19.851430 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9d5257-3c48-4448-b75e-43c1c7e9d07d None admin] RBAC: Adding query filter params (role.id=9db5671c65d3414ea2682afa2ccf1403, scope.project.id=0c5d68529f48460f9f6e03e3f8fb205b, user.id=4105c099a7824283aa03a6c3cc324850) {{(pid=80539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 11:57:19.852238 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9d5257-3c48-4448-b75e-43c1c7e9d07d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:57:19.855524 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9d5257-3c48-4448-b75e-43c1c7e9d07d None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:57:19.869456 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 98/201] 10.222.0.23 () {56 vars in 1374 bytes} [Thu May 19 11:57:19 2022] GET /identity/v3/role_assignments?user.id=4105c099a7824283aa03a6c3cc324850&scope.project.id=0c5d68529f48460f9f6e03e3f8fb205b&role.id=9db5671c65d3414ea2682afa2ccf1403 => generated 259 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:57:19.870611 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9d5257-3c48-4448-b75e-43c1c7e9d07d None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:57:20.004250 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 104/202] 10.222.0.23 () {58 vars in 1282 bytes} [Thu May 19 11:57:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 546 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 11:57:20.010751 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e99978-90ef-4cb1-b9d9-06704250a33c None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:57:20.108590 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9d5257-3c48-4448-b75e-43c1c7e9d07d None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:57:20.132243 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:57:20.142762 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:57:20.155624 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e93344c-2baa-41db-b774-dcd5e5b34981 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:57:20.157409 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e93344c-2baa-41db-b774-dcd5e5b34981 None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:20.157703 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e93344c-2baa-41db-b774-dcd5e5b34981 None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:20.157991 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e93344c-2baa-41db-b774-dcd5e5b34981 None admin] PATH_INFO: `/v3/users/a92936fa37984cfda4b8a7a5cc363e6e` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:20.159461 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e93344c-2baa-41db-b774-dcd5e5b34981 None admin] RBAC: Authorizing `identity:get_user(user_id=a92936fa37984cfda4b8a7a5cc363e6e)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:57:20.161052 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e93344c-2baa-41db-b774-dcd5e5b34981 None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:57:20.162857 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 99/203] 10.222.0.23 () {56 vars in 1177 bytes} [Thu May 19 11:57:19 2022] GET /identity/v3/users/a92936fa37984cfda4b8a7a5cc363e6e => generated 289 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:57:20.170817 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e93344c-2baa-41db-b774-dcd5e5b34981 None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:57:20.180903 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e99978-90ef-4cb1-b9d9-06704250a33c None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:57:20.274349 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956f8e19-7ef2-4df1-8af9-1b2600293b27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:57:20.275150 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956f8e19-7ef2-4df1-8af9-1b2600293b27 None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:20.275441 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956f8e19-7ef2-4df1-8af9-1b2600293b27 None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:20.275724 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956f8e19-7ef2-4df1-8af9-1b2600293b27 None admin] PATH_INFO: `/v3/projects/0c5d68529f48460f9f6e03e3f8fb205b` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:20.276969 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956f8e19-7ef2-4df1-8af9-1b2600293b27 None admin] RBAC: Authorizing `identity:get_project(project_id=0c5d68529f48460f9f6e03e3f8fb205b)` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:57:20.278458 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956f8e19-7ef2-4df1-8af9-1b2600293b27 None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:57:20.288742 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 105/204] 10.222.0.23 () {56 vars in 1186 bytes} [Thu May 19 11:57:20 2022] GET /identity/v3/projects/0c5d68529f48460f9f6e03e3f8fb205b => generated 302 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:57:20.294516 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956f8e19-7ef2-4df1-8af9-1b2600293b27 None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:57:20.303535 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e93344c-2baa-41db-b774-dcd5e5b34981 None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:57:20.378844 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d3219e-7a22-4431-a746-1305acc7b4f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:57:20.379530 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d3219e-7a22-4431-a746-1305acc7b4f9 None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:20.380128 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d3219e-7a22-4431-a746-1305acc7b4f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:20.380413 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d3219e-7a22-4431-a746-1305acc7b4f9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:20.489554 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:57:20.497733 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d3219e-7a22-4431-a746-1305acc7b4f9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:57:20.499230 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d3219e-7a22-4431-a746-1305acc7b4f9 None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:57:20.527405 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956f8e19-7ef2-4df1-8af9-1b2600293b27 None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:57:20.611842 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 100/205] 10.222.0.23 () {58 vars in 1282 bytes} [Thu May 19 11:57:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 442 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 19 11:57:20.617728 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d3219e-7a22-4431-a746-1305acc7b4f9 None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:57:20.625156 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fdfde9f-7845-4a91-9913-95253d29bcb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:57:20.625839 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdfde9f-7845-4a91-9913-95253d29bcb4 None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:20.626128 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdfde9f-7845-4a91-9913-95253d29bcb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:20.626410 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdfde9f-7845-4a91-9913-95253d29bcb4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:20.627380 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fdfde9f-7845-4a91-9913-95253d29bcb4 None admin] RBAC: Adding query filter params (role.id=8215583bb4224c75b093ae8e3993eb9f, scope.project.id=0c5d68529f48460f9f6e03e3f8fb205b, user.id=4105c099a7824283aa03a6c3cc324850) {{(pid=80540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 11:57:20.627707 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fdfde9f-7845-4a91-9913-95253d29bcb4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:57:20.629262 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fdfde9f-7845-4a91-9913-95253d29bcb4 None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:57:20.631386 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:57:20.639813 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 106/206] 10.222.0.23 () {56 vars in 1374 bytes} [Thu May 19 11:57:20 2022] GET /identity/v3/role_assignments?user.id=4105c099a7824283aa03a6c3cc324850&scope.project.id=0c5d68529f48460f9f6e03e3f8fb205b&role.id=8215583bb4224c75b093ae8e3993eb9f => generated 259 bytes in 343 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 11:57:20.794062 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:57:20.806971 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d3219e-7a22-4431-a746-1305acc7b4f9 None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:57:20.858435 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1887e0ac-0d3e-4f75-9c30-5fba4b26114f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:57:20.860448 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1887e0ac-0d3e-4f75-9c30-5fba4b26114f None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:20.860939 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1887e0ac-0d3e-4f75-9c30-5fba4b26114f None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:20.861406 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1887e0ac-0d3e-4f75-9c30-5fba4b26114f None admin] PATH_INFO: `/v3/projects/64092131887b4859ae47a1f4c3e0d24a` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:20.863409 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1887e0ac-0d3e-4f75-9c30-5fba4b26114f None admin] RBAC: Authorizing `identity:get_project(project_id=64092131887b4859ae47a1f4c3e0d24a)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:57:20.865160 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1887e0ac-0d3e-4f75-9c30-5fba4b26114f None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:57:20.867107 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 101/207] 10.222.0.23 () {56 vars in 1186 bytes} [Thu May 19 11:57:20 2022] GET /identity/v3/projects/64092131887b4859ae47a1f4c3e0d24a => generated 298 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:57:20.873060 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fdfde9f-7845-4a91-9913-95253d29bcb4 None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:57:21.022660 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fdfde9f-7845-4a91-9913-95253d29bcb4 None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:57:21.073027 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51a49ae-c558-4df9-b226-d216521eb8ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:57:21.073706 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51a49ae-c558-4df9-b226-d216521eb8ae None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:21.073994 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51a49ae-c558-4df9-b226-d216521eb8ae None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:21.074266 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51a49ae-c558-4df9-b226-d216521eb8ae None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:21.075105 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51a49ae-c558-4df9-b226-d216521eb8ae None admin] RBAC: Adding query filter params (role.id=a49254f8009c463e88e5e0b3909c5904, scope.project.id=64092131887b4859ae47a1f4c3e0d24a, user.id=a92936fa37984cfda4b8a7a5cc363e6e) {{(pid=80540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 11:57:21.075421 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51a49ae-c558-4df9-b226-d216521eb8ae None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:57:21.076940 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51a49ae-c558-4df9-b226-d216521eb8ae None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:57:21.088690 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 107/208] 10.222.0.23 () {56 vars in 1374 bytes} [Thu May 19 11:57:20 2022] GET /identity/v3/role_assignments?user.id=a92936fa37984cfda4b8a7a5cc363e6e&scope.project.id=64092131887b4859ae47a1f4c3e0d24a&role.id=a49254f8009c463e88e5e0b3909c5904 => generated 608 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:57:21.965082 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f3064e-30aa-449b-af71-0001fd1476bb None None] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:21.965576 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f3064e-30aa-449b-af71-0001fd1476bb None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:21.965649 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f3064e-30aa-449b-af71-0001fd1476bb None None] PATH_INFO: `/` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:21.967188 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 102/209] 10.222.0.23 () {54 vars in 926 bytes} [Thu May 19 11:57:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 11:57:21.973418 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0c56c1-b11d-4bb8-9ed8-26e4f10a0082 None None] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:21.974409 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0c56c1-b11d-4bb8-9ed8-26e4f10a0082 None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:21.974903 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0c56c1-b11d-4bb8-9ed8-26e4f10a0082 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:21.994960 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.auth.core [None req-4e0c56c1-b11d-4bb8-9ed8-26e4f10a0082 None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 11:57:21.996611 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-4e0c56c1-b11d-4bb8-9ed8-26e4f10a0082 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 11:57:22.064141 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:57:22.076354 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 108/210] 10.222.0.23 () {58 vars in 1023 bytes} [Thu May 19 11:57:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 11:57:22.099491 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f3064e-30aa-449b-af71-0001fd1476bb None None] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:57:22.190182 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353322e1-40fa-4858-9702-86452d9b1778 None None] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:22.190505 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353322e1-40fa-4858-9702-86452d9b1778 None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:22.190750 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353322e1-40fa-4858-9702-86452d9b1778 None None] PATH_INFO: `/` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:22.193416 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 109/211] 10.222.0.23 () {54 vars in 926 bytes} [Thu May 19 11:57:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 11:57:22.199297 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cf57db-d6cd-4c45-a6cd-422549530e19 None None] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:22.199892 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cf57db-d6cd-4c45-a6cd-422549530e19 None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:22.200383 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cf57db-d6cd-4c45-a6cd-422549530e19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:22.210803 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f3064e-30aa-449b-af71-0001fd1476bb None None] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:57:22.220467 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.auth.core [None req-12cf57db-d6cd-4c45-a6cd-422549530e19 None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 11:57:22.222390 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-12cf57db-d6cd-4c45-a6cd-422549530e19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 11:57:22.248707 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249ed492-6739-458b-9f1c-2dab7abcce0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:57:22.250344 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249ed492-6739-458b-9f1c-2dab7abcce0d None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:22.250677 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249ed492-6739-458b-9f1c-2dab7abcce0d None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:22.250958 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249ed492-6739-458b-9f1c-2dab7abcce0d None admin] PATH_INFO: `/v3/roles/member` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:22.263151 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249ed492-6739-458b-9f1c-2dab7abcce0d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 11:57:22.271933 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application [None req-249ed492-6739-458b-9f1c-2dab7abcce0d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 11:57:22.271933 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 11:57:22.271933 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 11:57:22.271933 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 11:57:22.271933 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 11:57:22.271933 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 11:57:22.271933 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 11:57:22.271933 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 11:57:22.271933 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 11:57:22.271933 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 11:57:22.271933 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 11:57:22.271933 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 11:57:22.271933 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 11:57:22.271933 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 11:57:22.271933 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 11:57:22.271933 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 11:57:22.272909 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 11:57:22.272909 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 11:57:22.272909 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 11:57:22.272909 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 11:57:22.272909 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 11:57:22.272909 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 11:57:22.272909 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 11:57:22.272909 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 11:57:22.272909 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 11:57:22.272909 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.get_or_create( May 19 11:57:22.272909 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 11:57:22.272909 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application with Lock( May 19 11:57:22.272909 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 11:57:22.272909 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._enter() May 19 11:57:22.272909 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 11:57:22.272909 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 11:57:22.273707 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 11:57:22.273707 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.creator() May 19 11:57:22.273707 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 11:57:22.273707 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application created_value = creator( May 19 11:57:22.273707 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 11:57:22.273707 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 11:57:22.273707 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 11:57:22.273707 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 11:57:22.273707 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 11:57:22.273707 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 11:57:22.273707 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 11:57:22.273707 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application May 19 11:57:22.273707 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 103/212] 10.222.0.23 () {56 vars in 1099 bytes} [Thu May 19 11:57:22 2022] GET /identity/v3/roles/member => generated 84 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 11:57:22.279181 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249ed492-6739-458b-9f1c-2dab7abcce0d None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:57:22.306988 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:57:22.316657 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 110/213] 10.222.0.23 () {58 vars in 1023 bytes} [Thu May 19 11:57:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 11:57:22.342011 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12cf57db-d6cd-4c45-a6cd-422549530e19 None None] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:57:22.398010 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249ed492-6739-458b-9f1c-2dab7abcce0d None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:57:22.437869 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a2e774-0e66-44d9-b236-7226424a92a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:57:22.438709 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a2e774-0e66-44d9-b236-7226424a92a4 None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:22.439004 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a2e774-0e66-44d9-b236-7226424a92a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:22.439284 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a2e774-0e66-44d9-b236-7226424a92a4 None admin] PATH_INFO: `/v3/roles` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:22.440157 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a2e774-0e66-44d9-b236-7226424a92a4 None admin] RBAC: Adding query filter params (name=member) {{(pid=80539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 11:57:22.440639 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a2e774-0e66-44d9-b236-7226424a92a4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:57:22.442229 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a2e774-0e66-44d9-b236-7226424a92a4 None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:57:22.452478 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 104/214] 10.222.0.23 () {56 vars in 1101 bytes} [Thu May 19 11:57:22 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 11:57:22.458457 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a2e774-0e66-44d9-b236-7226424a92a4 None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:57:22.460954 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12cf57db-d6cd-4c45-a6cd-422549530e19 None None] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:57:22.486547 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:57:22.499244 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36ba0b6-3e63-4ac4-a2cc-eafc8a0f4611 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:57:22.501302 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36ba0b6-3e63-4ac4-a2cc-eafc8a0f4611 None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:22.501786 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36ba0b6-3e63-4ac4-a2cc-eafc8a0f4611 None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:22.502246 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36ba0b6-3e63-4ac4-a2cc-eafc8a0f4611 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:22.503711 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36ba0b6-3e63-4ac4-a2cc-eafc8a0f4611 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:57:22.505483 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36ba0b6-3e63-4ac4-a2cc-eafc8a0f4611 None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:57:22.512159 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 111/215] 10.222.0.23 () {56 vars in 1108 bytes} [Thu May 19 11:57:22 2022] GET /identity/v3/domains/default => generated 199 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:57:22.513401 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36ba0b6-3e63-4ac4-a2cc-eafc8a0f4611 None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:57:22.556681 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:57:22.574721 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a2e774-0e66-44d9-b236-7226424a92a4 None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:57:22.612641 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec58b4c-da5c-4532-aed1-18af875f310f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:57:22.613516 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec58b4c-da5c-4532-aed1-18af875f310f None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:22.613796 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec58b4c-da5c-4532-aed1-18af875f310f None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:22.614069 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec58b4c-da5c-4532-aed1-18af875f310f None admin] PATH_INFO: `/v3/users/2d4dca52a2a64b5cb2ac901fb59ecdc2` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:22.628838 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec58b4c-da5c-4532-aed1-18af875f310f None admin] RBAC: Authorizing `identity:get_user(user_id=2d4dca52a2a64b5cb2ac901fb59ecdc2)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:57:22.630375 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec58b4c-da5c-4532-aed1-18af875f310f None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:57:22.631371 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36ba0b6-3e63-4ac4-a2cc-eafc8a0f4611 None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:57:22.636616 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 105/216] 10.222.0.23 () {56 vars in 1177 bytes} [Thu May 19 11:57:22 2022] GET /identity/v3/users/2d4dca52a2a64b5cb2ac901fb59ecdc2 => generated 297 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 11:57:22.636616 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbaf78d-2db8-4c34-8602-f31c0e86a259 None None] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:22.636616 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbaf78d-2db8-4c34-8602-f31c0e86a259 None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:22.636616 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbaf78d-2db8-4c34-8602-f31c0e86a259 None None] PATH_INFO: `/` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:22.639349 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 106/217] 10.222.0.23 () {54 vars in 926 bytes} [Thu May 19 11:57:22 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 11:57:22.640376 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbaf78d-2db8-4c34-8602-f31c0e86a259 None None] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:57:22.674870 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb271b5-a08d-4b0e-9a51-451553585755 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:57:22.675559 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb271b5-a08d-4b0e-9a51-451553585755 None admin] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:22.675900 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb271b5-a08d-4b0e-9a51-451553585755 None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:22.676179 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb271b5-a08d-4b0e-9a51-451553585755 None admin] PATH_INFO: `/v3/users` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:22.676930 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb271b5-a08d-4b0e-9a51-451553585755 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:57:22.678406 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb271b5-a08d-4b0e-9a51-451553585755 None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:57:22.755676 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbaf78d-2db8-4c34-8602-f31c0e86a259 None None] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:57:22.792943 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c3b19b-4b99-4fe0-a70a-5193d7421a14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:57:22.793622 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c3b19b-4b99-4fe0-a70a-5193d7421a14 None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:22.793907 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c3b19b-4b99-4fe0-a70a-5193d7421a14 None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:22.794236 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c3b19b-4b99-4fe0-a70a-5193d7421a14 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:22.800042 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 112/218] 10.222.0.23 () {60 vars in 1132 bytes} [Thu May 19 11:57:22 2022] POST /identity/v3/users => generated 297 bytes in 287 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 11:57:22.801561 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64066e27-66e9-42ea-803e-2d73e5e34498 None None] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:22.801857 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64066e27-66e9-42ea-803e-2d73e5e34498 None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:22.802137 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64066e27-66e9-42ea-803e-2d73e5e34498 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:22.810026 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:57:22.846062 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.auth.core [None req-64066e27-66e9-42ea-803e-2d73e5e34498 None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 11:57:22.847332 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-64066e27-66e9-42ea-803e-2d73e5e34498 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 11:57:22.897172 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c3b19b-4b99-4fe0-a70a-5193d7421a14 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:57:22.899852 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c3b19b-4b99-4fe0-a70a-5193d7421a14 None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:57:22.910757 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:57:22.957973 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 113/219] 10.222.0.23 () {58 vars in 1023 bytes} [Thu May 19 11:57:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 11:57:22.980927 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64066e27-66e9-42ea-803e-2d73e5e34498 None None] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:57:23.039481 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 107/220] 10.222.0.23 () {58 vars in 1282 bytes} [Thu May 19 11:57:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 11:57:23.045166 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c3b19b-4b99-4fe0-a70a-5193d7421a14 None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:57:23.096905 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:57:23.122316 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64066e27-66e9-42ea-803e-2d73e5e34498 None None] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:57:23.169458 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb06c7b-9560-4707-8961-b42a261af384 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:57:23.170396 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb06c7b-9560-4707-8961-b42a261af384 None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:23.170698 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb06c7b-9560-4707-8961-b42a261af384 None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:23.171152 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb06c7b-9560-4707-8961-b42a261af384 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:23.177629 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c3b19b-4b99-4fe0-a70a-5193d7421a14 None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:57:23.188114 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb06c7b-9560-4707-8961-b42a261af384 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 11:57:23.197791 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application [None req-8bb06c7b-9560-4707-8961-b42a261af384 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 11:57:23.197791 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 11:57:23.197791 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 11:57:23.197791 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 11:57:23.197791 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 11:57:23.197791 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 11:57:23.197791 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 11:57:23.197791 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 11:57:23.197791 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 11:57:23.197791 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 11:57:23.197791 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 11:57:23.197791 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 11:57:23.197791 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 11:57:23.197791 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 11:57:23.197791 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 11:57:23.197791 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 11:57:23.198909 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 11:57:23.198909 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 11:57:23.198909 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 11:57:23.198909 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 11:57:23.198909 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 11:57:23.198909 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 11:57:23.198909 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 11:57:23.198909 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 11:57:23.198909 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 11:57:23.198909 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.get_or_create( May 19 11:57:23.198909 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 11:57:23.198909 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application with Lock( May 19 11:57:23.198909 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 11:57:23.198909 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._enter() May 19 11:57:23.198909 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 11:57:23.198909 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 11:57:23.199714 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 11:57:23.199714 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.creator() May 19 11:57:23.199714 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 11:57:23.199714 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application created_value = creator( May 19 11:57:23.199714 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 11:57:23.199714 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 11:57:23.199714 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 11:57:23.199714 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 11:57:23.199714 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 11:57:23.199714 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 11:57:23.199714 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 11:57:23.199714 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application May 19 11:57:23.199714 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 114/221] 10.222.0.23 () {56 vars in 1096 bytes} [Thu May 19 11:57:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 11:57:23.200475 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65d35bb-5681-480a-b304-75cc6655c2ab None None] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:23.200750 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65d35bb-5681-480a-b304-75cc6655c2ab None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:23.200931 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65d35bb-5681-480a-b304-75cc6655c2ab None None] PATH_INFO: `/` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:23.202228 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 115/222] 10.222.0.23 () {54 vars in 926 bytes} [Thu May 19 11:57:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 11:57:23.206428 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65d35bb-5681-480a-b304-75cc6655c2ab None None] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:57:23.229294 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948fede0-5f14-4355-820f-04d0c8310ac2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:57:23.230629 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948fede0-5f14-4355-820f-04d0c8310ac2 None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:23.231216 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948fede0-5f14-4355-820f-04d0c8310ac2 None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:23.231710 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948fede0-5f14-4355-820f-04d0c8310ac2 None admin] PATH_INFO: `/v3/projects/0c5d68529f48460f9f6e03e3f8fb205b` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:23.233198 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948fede0-5f14-4355-820f-04d0c8310ac2 None admin] RBAC: Authorizing `identity:get_project(project_id=0c5d68529f48460f9f6e03e3f8fb205b)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:57:23.235836 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948fede0-5f14-4355-820f-04d0c8310ac2 None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:57:23.238743 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 108/223] 10.222.0.23 () {56 vars in 1186 bytes} [Thu May 19 11:57:23 2022] GET /identity/v3/projects/0c5d68529f48460f9f6e03e3f8fb205b => generated 302 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:57:23.239953 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75d7d3e-ae3c-4c50-b29c-3ae81707c2ea None None] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:23.240375 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75d7d3e-ae3c-4c50-b29c-3ae81707c2ea None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:23.240653 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75d7d3e-ae3c-4c50-b29c-3ae81707c2ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:23.259217 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.auth.core [None req-f75d7d3e-ae3c-4c50-b29c-3ae81707c2ea None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 11:57:23.259811 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:57:23.261399 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-f75d7d3e-ae3c-4c50-b29c-3ae81707c2ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 11:57:23.348146 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65d35bb-5681-480a-b304-75cc6655c2ab None None] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:57:23.353679 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 109/224] 10.222.0.23 () {58 vars in 1023 bytes} [Thu May 19 11:57:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 11:57:23.354364 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75d7d3e-ae3c-4c50-b29c-3ae81707c2ea None None] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:57:23.398659 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04453436-e1c8-4134-8e19-f3cc1942caa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:57:23.399367 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04453436-e1c8-4134-8e19-f3cc1942caa5 None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:23.399655 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04453436-e1c8-4134-8e19-f3cc1942caa5 None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:23.399967 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04453436-e1c8-4134-8e19-f3cc1942caa5 None admin] PATH_INFO: `/v3/roles` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:23.400971 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:57:23.401858 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04453436-e1c8-4134-8e19-f3cc1942caa5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 11:57:23.402169 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04453436-e1c8-4134-8e19-f3cc1942caa5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:57:23.404283 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04453436-e1c8-4134-8e19-f3cc1942caa5 None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:57:23.419882 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 116/225] 10.222.0.23 () {56 vars in 1099 bytes} [Thu May 19 11:57:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:57:23.420796 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04453436-e1c8-4134-8e19-f3cc1942caa5 None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:57:23.477341 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75d7d3e-ae3c-4c50-b29c-3ae81707c2ea None None] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:57:23.515155 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d232ec-ced5-4ff2-923e-b0860b911232 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:57:23.516839 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d232ec-ced5-4ff2-923e-b0860b911232 None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:23.517125 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d232ec-ced5-4ff2-923e-b0860b911232 None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:23.517665 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d232ec-ced5-4ff2-923e-b0860b911232 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:23.518516 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d232ec-ced5-4ff2-923e-b0860b911232 None admin] RBAC: Adding query filter params (role.id=259eb400cdb4439db72d14456ae9cd08, scope.project.id=0c5d68529f48460f9f6e03e3f8fb205b, user.id=2d4dca52a2a64b5cb2ac901fb59ecdc2) {{(pid=80539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 11:57:23.518831 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d232ec-ced5-4ff2-923e-b0860b911232 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:57:23.520756 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d232ec-ced5-4ff2-923e-b0860b911232 None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:57:23.529144 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 110/226] 10.222.0.23 () {56 vars in 1374 bytes} [Thu May 19 11:57:23 2022] GET /identity/v3/role_assignments?user.id=2d4dca52a2a64b5cb2ac901fb59ecdc2&scope.project.id=0c5d68529f48460f9f6e03e3f8fb205b&role.id=259eb400cdb4439db72d14456ae9cd08 => generated 259 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:57:23.529997 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d232ec-ced5-4ff2-923e-b0860b911232 None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:57:23.586129 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:57:23.590995 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04453436-e1c8-4134-8e19-f3cc1942caa5 None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:57:23.686288 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef6e08e-1dd2-4dfd-bdbe-287411f7f26f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:57:23.687655 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef6e08e-1dd2-4dfd-bdbe-287411f7f26f None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:23.688180 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef6e08e-1dd2-4dfd-bdbe-287411f7f26f None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:23.688475 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef6e08e-1dd2-4dfd-bdbe-287411f7f26f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:23.712857 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef6e08e-1dd2-4dfd-bdbe-287411f7f26f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 11:57:23.732096 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application [None req-aef6e08e-1dd2-4dfd-bdbe-287411f7f26f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 11:57:23.732096 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 11:57:23.732096 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 11:57:23.732096 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 11:57:23.732096 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 11:57:23.732096 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 11:57:23.732096 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 11:57:23.732096 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 11:57:23.732096 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 11:57:23.732096 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 11:57:23.732096 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 11:57:23.732096 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 11:57:23.732096 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 11:57:23.732096 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 11:57:23.732096 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 11:57:23.732096 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 11:57:23.733042 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 11:57:23.733042 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 11:57:23.733042 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 11:57:23.733042 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 11:57:23.733042 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 11:57:23.733042 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 11:57:23.733042 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 11:57:23.733042 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 11:57:23.733042 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 11:57:23.733042 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.get_or_create( May 19 11:57:23.733042 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 11:57:23.733042 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application with Lock( May 19 11:57:23.733042 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 11:57:23.733042 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._enter() May 19 11:57:23.733042 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 11:57:23.733042 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 11:57:23.734476 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 11:57:23.734476 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.creator() May 19 11:57:23.734476 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 11:57:23.734476 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application created_value = creator( May 19 11:57:23.734476 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 11:57:23.734476 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 11:57:23.734476 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 11:57:23.734476 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 11:57:23.734476 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 11:57:23.734476 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 11:57:23.734476 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 11:57:23.734476 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application May 19 11:57:23.736765 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 117/227] 10.222.0.23 () {56 vars in 1114 bytes} [Thu May 19 11:57:23 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 316 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 11:57:23.740864 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d232ec-ced5-4ff2-923e-b0860b911232 None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:57:23.743750 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:57:23.745386 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef6e08e-1dd2-4dfd-bdbe-287411f7f26f None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:57:23.837555 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0ebcfd-5d7f-47b5-b3dd-947c3a7f4ca6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:57:23.838802 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0ebcfd-5d7f-47b5-b3dd-947c3a7f4ca6 None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:23.839229 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0ebcfd-5d7f-47b5-b3dd-947c3a7f4ca6 None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:23.839653 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0ebcfd-5d7f-47b5-b3dd-947c3a7f4ca6 None admin] PATH_INFO: `/v3/users/4105c099a7824283aa03a6c3cc324850` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:23.844588 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0ebcfd-5d7f-47b5-b3dd-947c3a7f4ca6 None admin] RBAC: Authorizing `identity:get_user(user_id=4105c099a7824283aa03a6c3cc324850)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:57:23.846471 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0ebcfd-5d7f-47b5-b3dd-947c3a7f4ca6 None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:57:23.848670 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 111/228] 10.222.0.23 () {56 vars in 1177 bytes} [Thu May 19 11:57:23 2022] GET /identity/v3/users/4105c099a7824283aa03a6c3cc324850 => generated 283 bytes in 319 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:57:23.866623 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0ebcfd-5d7f-47b5-b3dd-947c3a7f4ca6 None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:57:23.889850 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef6e08e-1dd2-4dfd-bdbe-287411f7f26f None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:57:23.940625 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4279d1-c4ce-49ea-bb95-643ed2f30f8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:57:23.941247 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4279d1-c4ce-49ea-bb95-643ed2f30f8c None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:23.941541 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4279d1-c4ce-49ea-bb95-643ed2f30f8c None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:23.941820 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4279d1-c4ce-49ea-bb95-643ed2f30f8c None admin] PATH_INFO: `/v3/roles` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:23.942592 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c4279d1-c4ce-49ea-bb95-643ed2f30f8c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 11:57:23.942901 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c4279d1-c4ce-49ea-bb95-643ed2f30f8c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:57:23.944356 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c4279d1-c4ce-49ea-bb95-643ed2f30f8c None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:57:23.977834 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 118/229] 10.222.0.23 () {56 vars in 1111 bytes} [Thu May 19 11:57:23 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:57:23.996860 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4279d1-c4ce-49ea-bb95-643ed2f30f8c None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:57:24.028700 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0ebcfd-5d7f-47b5-b3dd-947c3a7f4ca6 None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:57:24.052280 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:57:24.104346 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bdc1047-333a-444d-a73f-09e170bb4c27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:57:24.105226 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdc1047-333a-444d-a73f-09e170bb4c27 None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:24.105506 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdc1047-333a-444d-a73f-09e170bb4c27 None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:24.105787 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdc1047-333a-444d-a73f-09e170bb4c27 None admin] PATH_INFO: `/v3/projects/0c5d68529f48460f9f6e03e3f8fb205b` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:24.110874 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bdc1047-333a-444d-a73f-09e170bb4c27 None admin] RBAC: Authorizing `identity:get_project(project_id=0c5d68529f48460f9f6e03e3f8fb205b)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:57:24.113510 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bdc1047-333a-444d-a73f-09e170bb4c27 None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:57:24.116942 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 112/230] 10.222.0.23 () {56 vars in 1186 bytes} [Thu May 19 11:57:23 2022] GET /identity/v3/projects/0c5d68529f48460f9f6e03e3f8fb205b => generated 302 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 11:57:24.141557 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bdc1047-333a-444d-a73f-09e170bb4c27 None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:57:24.175620 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4279d1-c4ce-49ea-bb95-643ed2f30f8c None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:57:24.221305 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95782a4-beed-42f5-8679-bbc122e84b66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:57:24.223547 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95782a4-beed-42f5-8679-bbc122e84b66 None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:24.225251 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95782a4-beed-42f5-8679-bbc122e84b66 None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:24.225996 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95782a4-beed-42f5-8679-bbc122e84b66 None admin] PATH_INFO: `/v3/users/4105c099a7824283aa03a6c3cc324850` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:24.229212 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95782a4-beed-42f5-8679-bbc122e84b66 None admin] RBAC: Authorizing `identity:get_user(user_id=4105c099a7824283aa03a6c3cc324850)` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:57:24.230915 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95782a4-beed-42f5-8679-bbc122e84b66 None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:57:24.236850 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 119/231] 10.222.0.23 () {56 vars in 1177 bytes} [Thu May 19 11:57:23 2022] GET /identity/v3/users/4105c099a7824283aa03a6c3cc324850 => generated 283 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:57:24.242255 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95782a4-beed-42f5-8679-bbc122e84b66 None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:57:24.250769 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:57:24.375826 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:57:24.391755 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95782a4-beed-42f5-8679-bbc122e84b66 None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:57:24.422561 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bdc1047-333a-444d-a73f-09e170bb4c27 None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:57:24.438218 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b7ece7-7c77-4199-9f5a-6216df397a6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:57:24.439450 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b7ece7-7c77-4199-9f5a-6216df397a6a None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:24.442199 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b7ece7-7c77-4199-9f5a-6216df397a6a None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:24.442732 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b7ece7-7c77-4199-9f5a-6216df397a6a None admin] PATH_INFO: `/v3/projects/0c5d68529f48460f9f6e03e3f8fb205b` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:24.446718 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b7ece7-7c77-4199-9f5a-6216df397a6a None admin] RBAC: Authorizing `identity:get_project(project_id=0c5d68529f48460f9f6e03e3f8fb205b)` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:57:24.448484 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b7ece7-7c77-4199-9f5a-6216df397a6a None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:57:24.452012 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 120/232] 10.222.0.23 () {56 vars in 1186 bytes} [Thu May 19 11:57:24 2022] GET /identity/v3/projects/0c5d68529f48460f9f6e03e3f8fb205b => generated 302 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 11:57:24.458119 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b7ece7-7c77-4199-9f5a-6216df397a6a None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:57:24.512974 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67f4527-f346-460e-a2ca-5a894844d165 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:57:24.513635 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67f4527-f346-460e-a2ca-5a894844d165 None admin] REQUEST_METHOD: `PUT` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:24.513916 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67f4527-f346-460e-a2ca-5a894844d165 None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:24.514373 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67f4527-f346-460e-a2ca-5a894844d165 None admin] PATH_INFO: `/v3/projects/0c5d68529f48460f9f6e03e3f8fb205b/users/4105c099a7824283aa03a6c3cc324850/roles/9db5671c65d3414ea2682afa2ccf1403` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:24.528250 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67f4527-f346-460e-a2ca-5a894844d165 None admin] RBAC: Authorizing `identity:create_grant(project_id=0c5d68529f48460f9f6e03e3f8fb205b, user_id=4105c099a7824283aa03a6c3cc324850, role_id=9db5671c65d3414ea2682afa2ccf1403)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:57:24.528250 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67f4527-f346-460e-a2ca-5a894844d165 None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:57:24.607730 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b7ece7-7c77-4199-9f5a-6216df397a6a None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:57:24.646428 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 113/233] 10.222.0.23 () {58 vars in 1439 bytes} [Thu May 19 11:57:24 2022] PUT /identity/v3/projects/0c5d68529f48460f9f6e03e3f8fb205b/users/4105c099a7824283aa03a6c3cc324850/roles/9db5671c65d3414ea2682afa2ccf1403 => generated 0 bytes in 510 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 11:57:24.647067 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5239b1b-e384-4b73-8eb1-edb29477d3e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:57:24.648429 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5239b1b-e384-4b73-8eb1-edb29477d3e3 None admin] REQUEST_METHOD: `PUT` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:24.649024 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5239b1b-e384-4b73-8eb1-edb29477d3e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:24.649696 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5239b1b-e384-4b73-8eb1-edb29477d3e3 None admin] PATH_INFO: `/v3/projects/0c5d68529f48460f9f6e03e3f8fb205b/users/4105c099a7824283aa03a6c3cc324850/roles/8215583bb4224c75b093ae8e3993eb9f` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:24.652769 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5239b1b-e384-4b73-8eb1-edb29477d3e3 None admin] RBAC: Authorizing `identity:create_grant(project_id=0c5d68529f48460f9f6e03e3f8fb205b, user_id=4105c099a7824283aa03a6c3cc324850, role_id=8215583bb4224c75b093ae8e3993eb9f)` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:57:24.654641 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5239b1b-e384-4b73-8eb1-edb29477d3e3 None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:57:24.784291 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 121/234] 10.222.0.23 () {58 vars in 1439 bytes} [Thu May 19 11:57:24 2022] PUT /identity/v3/projects/0c5d68529f48460f9f6e03e3f8fb205b/users/4105c099a7824283aa03a6c3cc324850/roles/8215583bb4224c75b093ae8e3993eb9f => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 11:57:25.449095 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23060fdf-dd9f-437f-9029-eaaa2b7b335f None None] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:25.449614 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23060fdf-dd9f-437f-9029-eaaa2b7b335f None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:25.449689 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23060fdf-dd9f-437f-9029-eaaa2b7b335f None None] PATH_INFO: `/` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:25.451022 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 114/235] 10.222.0.23 () {54 vars in 926 bytes} [Thu May 19 11:57:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 11:57:25.457250 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b29ff8c-95ce-4191-b6a9-8d470de2c9d9 None None] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:25.458168 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b29ff8c-95ce-4191-b6a9-8d470de2c9d9 None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:25.458654 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b29ff8c-95ce-4191-b6a9-8d470de2c9d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:25.481302 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.auth.core [None req-9b29ff8c-95ce-4191-b6a9-8d470de2c9d9 None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 11:57:25.482862 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-9b29ff8c-95ce-4191-b6a9-8d470de2c9d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 11:57:25.501018 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:57:25.589175 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 122/236] 10.222.0.23 () {58 vars in 1023 bytes} [Thu May 19 11:57:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 11:57:25.624374 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23060fdf-dd9f-437f-9029-eaaa2b7b335f None None] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:57:25.680199 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f23c0dc-36be-4034-91ff-1b19582a0cdd None None] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:25.680518 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f23c0dc-36be-4034-91ff-1b19582a0cdd None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:25.680763 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f23c0dc-36be-4034-91ff-1b19582a0cdd None None] PATH_INFO: `/` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:25.682359 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 123/237] 10.222.0.23 () {54 vars in 926 bytes} [Thu May 19 11:57:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 11:57:25.688223 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b863ff-ce84-4e96-82ae-fd8cf75b6eba None None] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:25.688697 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b863ff-ce84-4e96-82ae-fd8cf75b6eba None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:25.689546 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b863ff-ce84-4e96-82ae-fd8cf75b6eba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:25.726060 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.auth.core [None req-c6b863ff-ce84-4e96-82ae-fd8cf75b6eba None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 11:57:25.727197 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-c6b863ff-ce84-4e96-82ae-fd8cf75b6eba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 11:57:25.808152 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:57:25.813079 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23060fdf-dd9f-437f-9029-eaaa2b7b335f None None] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:57:25.864344 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 124/238] 10.222.0.23 () {58 vars in 1023 bytes} [Thu May 19 11:57:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 11:57:25.865612 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afd3647-921a-466d-9e50-f8358058d76f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:57:25.866472 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afd3647-921a-466d-9e50-f8358058d76f None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:25.866757 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afd3647-921a-466d-9e50-f8358058d76f None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:25.867213 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afd3647-921a-466d-9e50-f8358058d76f None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:25.887056 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4afd3647-921a-466d-9e50-f8358058d76f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 11:57:25.887357 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b863ff-ce84-4e96-82ae-fd8cf75b6eba None None] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:57:25.895632 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application [None req-4afd3647-921a-466d-9e50-f8358058d76f None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 19 11:57:25.895632 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 11:57:25.895632 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 11:57:25.895632 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 11:57:25.895632 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 11:57:25.895632 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 11:57:25.895632 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 11:57:25.895632 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 11:57:25.895632 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 11:57:25.895632 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 11:57:25.895632 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 11:57:25.895632 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 11:57:25.895632 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 11:57:25.895632 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 11:57:25.895632 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 11:57:25.895632 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 11:57:25.896646 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 11:57:25.896646 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 11:57:25.896646 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 11:57:25.896646 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 11:57:25.896646 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 11:57:25.896646 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 11:57:25.896646 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 11:57:25.896646 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 11:57:25.896646 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 11:57:25.896646 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.get_or_create( May 19 11:57:25.896646 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 11:57:25.896646 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application with Lock( May 19 11:57:25.896646 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 11:57:25.896646 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._enter() May 19 11:57:25.896646 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 11:57:25.896646 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 11:57:25.897421 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 11:57:25.897421 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.creator() May 19 11:57:25.897421 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 11:57:25.897421 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application created_value = creator( May 19 11:57:25.897421 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 11:57:25.897421 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 11:57:25.897421 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 11:57:25.897421 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 11:57:25.897421 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 11:57:25.897421 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 11:57:25.897421 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 19 11:57:25.897421 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application May 19 11:57:25.897421 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 115/239] 10.222.0.23 () {56 vars in 1099 bytes} [Thu May 19 11:57:25 2022] GET /identity/v3/roles/reader => generated 84 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 11:57:25.904723 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afd3647-921a-466d-9e50-f8358058d76f None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:57:26.021696 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:57:26.050479 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b863ff-ce84-4e96-82ae-fd8cf75b6eba None None] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:57:26.097902 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2bed655-88f9-4312-9dec-225bc42c4156 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:57:26.099064 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bed655-88f9-4312-9dec-225bc42c4156 None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:26.099566 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bed655-88f9-4312-9dec-225bc42c4156 None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:26.100058 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bed655-88f9-4312-9dec-225bc42c4156 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:26.102266 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2bed655-88f9-4312-9dec-225bc42c4156 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:57:26.103979 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2bed655-88f9-4312-9dec-225bc42c4156 None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:57:26.107446 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 125/240] 10.222.0.23 () {56 vars in 1108 bytes} [Thu May 19 11:57:25 2022] GET /identity/v3/domains/default => generated 199 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:57:26.113276 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2bed655-88f9-4312-9dec-225bc42c4156 None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:57:26.146976 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afd3647-921a-466d-9e50-f8358058d76f None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:57:26.198617 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d29b80c-6d23-4d11-843d-d4fe5b162108 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:57:26.199656 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d29b80c-6d23-4d11-843d-d4fe5b162108 None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:26.200235 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d29b80c-6d23-4d11-843d-d4fe5b162108 None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:26.200804 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d29b80c-6d23-4d11-843d-d4fe5b162108 None admin] PATH_INFO: `/v3/roles` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:26.201787 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d29b80c-6d23-4d11-843d-d4fe5b162108 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 11:57:26.202304 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d29b80c-6d23-4d11-843d-d4fe5b162108 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:57:26.204763 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d29b80c-6d23-4d11-843d-d4fe5b162108 None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:57:26.214600 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 116/241] 10.222.0.23 () {56 vars in 1101 bytes} [Thu May 19 11:57:25 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:57:26.216359 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade7a085-e302-40cf-afe4-7ac8d965b1cb None None] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:26.216919 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade7a085-e302-40cf-afe4-7ac8d965b1cb None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:26.217414 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade7a085-e302-40cf-afe4-7ac8d965b1cb None None] PATH_INFO: `/` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:26.219714 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 117/242] 10.222.0.23 () {54 vars in 926 bytes} [Thu May 19 11:57:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 11:57:26.220698 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade7a085-e302-40cf-afe4-7ac8d965b1cb None None] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:57:26.252624 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:57:26.258969 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2bed655-88f9-4312-9dec-225bc42c4156 None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:57:26.306562 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ca42a7-2598-4fc5-b2dc-550f838a1219 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:57:26.307243 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ca42a7-2598-4fc5-b2dc-550f838a1219 None admin] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:26.307526 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ca42a7-2598-4fc5-b2dc-550f838a1219 None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:26.307832 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ca42a7-2598-4fc5-b2dc-550f838a1219 None admin] PATH_INFO: `/v3/users` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:26.308574 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ca42a7-2598-4fc5-b2dc-550f838a1219 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:57:26.310316 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ca42a7-2598-4fc5-b2dc-550f838a1219 None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:57:26.346541 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade7a085-e302-40cf-afe4-7ac8d965b1cb None None] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:57:26.385464 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6a8df7-7a32-448d-b388-75e4648571e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:57:26.386336 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6a8df7-7a32-448d-b388-75e4648571e9 None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:26.386616 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6a8df7-7a32-448d-b388-75e4648571e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:26.386886 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6a8df7-7a32-448d-b388-75e4648571e9 None admin] PATH_INFO: `/v3/users/0b74c769188e46608374f314768d312a` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:26.401170 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6a8df7-7a32-448d-b388-75e4648571e9 None admin] RBAC: Authorizing `identity:get_user(user_id=0b74c769188e46608374f314768d312a)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:57:26.402737 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6a8df7-7a32-448d-b388-75e4648571e9 None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:57:26.404497 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 118/243] 10.222.0.23 () {56 vars in 1177 bytes} [Thu May 19 11:57:26 2022] GET /identity/v3/users/0b74c769188e46608374f314768d312a => generated 297 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:57:26.410162 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a371d9a-889e-482c-8d1d-eb816de0604c None None] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:26.410162 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a371d9a-889e-482c-8d1d-eb816de0604c None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:26.410162 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a371d9a-889e-482c-8d1d-eb816de0604c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:26.426618 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.auth.core [None req-0a371d9a-889e-482c-8d1d-eb816de0604c None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 11:57:26.427842 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-0a371d9a-889e-482c-8d1d-eb816de0604c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 11:57:26.428172 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:57:26.440464 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 126/244] 10.222.0.23 () {60 vars in 1132 bytes} [Thu May 19 11:57:26 2022] POST /identity/v3/users => generated 293 bytes in 327 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 11:57:26.444099 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ca42a7-2598-4fc5-b2dc-550f838a1219 None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:57:26.587600 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 119/245] 10.222.0.23 () {58 vars in 1023 bytes} [Thu May 19 11:57:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 11:57:26.612056 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a371d9a-889e-482c-8d1d-eb816de0604c None None] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:57:26.632632 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:57:26.686483 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ca42a7-2598-4fc5-b2dc-550f838a1219 None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:57:26.730994 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6518819e-981e-439b-93fe-22af7d8ce8f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:57:26.730994 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6518819e-981e-439b-93fe-22af7d8ce8f8 None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:26.731407 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6518819e-981e-439b-93fe-22af7d8ce8f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:26.731605 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6518819e-981e-439b-93fe-22af7d8ce8f8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:26.803954 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a371d9a-889e-482c-8d1d-eb816de0604c None None] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:57:26.842499 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:57:26.845147 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea84a242-8b75-44e2-bba9-2b639041aa38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:57:26.845990 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6518819e-981e-439b-93fe-22af7d8ce8f8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:57:26.846458 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea84a242-8b75-44e2-bba9-2b639041aa38 None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:26.846938 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea84a242-8b75-44e2-bba9-2b639041aa38 None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:26.847514 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea84a242-8b75-44e2-bba9-2b639041aa38 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:26.847616 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6518819e-981e-439b-93fe-22af7d8ce8f8 None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:57:26.860021 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea84a242-8b75-44e2-bba9-2b639041aa38 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 11:57:26.870066 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application [None req-ea84a242-8b75-44e2-bba9-2b639041aa38 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 11:57:26.870066 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 11:57:26.870066 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 11:57:26.870066 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 11:57:26.870066 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 11:57:26.870066 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 11:57:26.870066 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 11:57:26.870066 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 11:57:26.870066 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 11:57:26.870066 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 11:57:26.870066 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 11:57:26.870066 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 11:57:26.870066 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 11:57:26.870066 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 11:57:26.870066 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 11:57:26.870066 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 11:57:26.871007 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 11:57:26.871007 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 11:57:26.871007 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 11:57:26.871007 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 11:57:26.871007 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 11:57:26.871007 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 11:57:26.871007 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 11:57:26.871007 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 11:57:26.871007 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 11:57:26.871007 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.get_or_create( May 19 11:57:26.871007 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 11:57:26.871007 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application with Lock( May 19 11:57:26.871007 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 11:57:26.871007 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._enter() May 19 11:57:26.871007 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 11:57:26.871007 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 11:57:26.871777 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 11:57:26.871777 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.creator() May 19 11:57:26.871777 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 11:57:26.871777 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application created_value = creator( May 19 11:57:26.871777 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 11:57:26.871777 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 11:57:26.871777 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 11:57:26.871777 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 11:57:26.871777 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 11:57:26.871777 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 11:57:26.871777 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 11:57:26.871777 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application May 19 11:57:26.874636 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 120/246] 10.222.0.23 () {56 vars in 1099 bytes} [Thu May 19 11:57:26 2022] GET /identity/v3/roles/member => generated 84 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 11:57:26.881348 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea84a242-8b75-44e2-bba9-2b639041aa38 None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:57:26.966978 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:57:26.977701 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 127/247] 10.222.0.23 () {58 vars in 1282 bytes} [Thu May 19 11:57:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 537 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 11:57:26.983368 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6518819e-981e-439b-93fe-22af7d8ce8f8 None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:57:27.002672 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea84a242-8b75-44e2-bba9-2b639041aa38 None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:57:27.041442 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2678bd63-52d7-4335-9fd5-33b0f61d8b30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:57:27.042464 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2678bd63-52d7-4335-9fd5-33b0f61d8b30 None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:27.042968 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2678bd63-52d7-4335-9fd5-33b0f61d8b30 None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:27.043443 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2678bd63-52d7-4335-9fd5-33b0f61d8b30 None admin] PATH_INFO: `/v3/roles` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:27.044466 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2678bd63-52d7-4335-9fd5-33b0f61d8b30 None admin] RBAC: Adding query filter params (name=member) {{(pid=80539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 11:57:27.044997 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2678bd63-52d7-4335-9fd5-33b0f61d8b30 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:57:27.047413 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2678bd63-52d7-4335-9fd5-33b0f61d8b30 None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:57:27.058221 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 121/248] 10.222.0.23 () {56 vars in 1101 bytes} [Thu May 19 11:57:26 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 11:57:27.073616 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2678bd63-52d7-4335-9fd5-33b0f61d8b30 None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:57:27.102094 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:57:27.105922 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6518819e-981e-439b-93fe-22af7d8ce8f8 None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:57:27.147981 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16408a00-613a-4efa-8684-ab693e648875 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:57:27.149400 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16408a00-613a-4efa-8684-ab693e648875 None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:27.149400 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16408a00-613a-4efa-8684-ab693e648875 None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:27.149571 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16408a00-613a-4efa-8684-ab693e648875 None admin] PATH_INFO: `/v3/projects/0c5d68529f48460f9f6e03e3f8fb205b` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:27.150853 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16408a00-613a-4efa-8684-ab693e648875 None admin] RBAC: Authorizing `identity:get_project(project_id=0c5d68529f48460f9f6e03e3f8fb205b)` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:57:27.152465 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16408a00-613a-4efa-8684-ab693e648875 None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:57:27.154510 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 128/249] 10.222.0.23 () {56 vars in 1186 bytes} [Thu May 19 11:57:26 2022] GET /identity/v3/projects/0c5d68529f48460f9f6e03e3f8fb205b => generated 302 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 11:57:27.161134 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16408a00-613a-4efa-8684-ab693e648875 None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:57:27.188245 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2678bd63-52d7-4335-9fd5-33b0f61d8b30 None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:57:27.221622 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:57:27.226902 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f36d530-cfc4-4cfb-9ca5-0598fe920a09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:57:27.227954 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f36d530-cfc4-4cfb-9ca5-0598fe920a09 None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:27.228290 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f36d530-cfc4-4cfb-9ca5-0598fe920a09 None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:27.228643 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f36d530-cfc4-4cfb-9ca5-0598fe920a09 None admin] PATH_INFO: `/v3/users/2d4dca52a2a64b5cb2ac901fb59ecdc2` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:27.230107 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f36d530-cfc4-4cfb-9ca5-0598fe920a09 None admin] RBAC: Authorizing `identity:get_user(user_id=2d4dca52a2a64b5cb2ac901fb59ecdc2)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:57:27.231689 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f36d530-cfc4-4cfb-9ca5-0598fe920a09 None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:57:27.233550 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 122/250] 10.222.0.23 () {56 vars in 1177 bytes} [Thu May 19 11:57:27 2022] GET /identity/v3/users/2d4dca52a2a64b5cb2ac901fb59ecdc2 => generated 297 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 11:57:27.239415 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f36d530-cfc4-4cfb-9ca5-0598fe920a09 None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:57:27.289282 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16408a00-613a-4efa-8684-ab693e648875 None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:57:27.333513 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28336b92-1e51-4a19-b6a6-f9d7c5aa95da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:57:27.334614 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28336b92-1e51-4a19-b6a6-f9d7c5aa95da None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:27.335092 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28336b92-1e51-4a19-b6a6-f9d7c5aa95da None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:27.335555 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28336b92-1e51-4a19-b6a6-f9d7c5aa95da None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:27.336649 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28336b92-1e51-4a19-b6a6-f9d7c5aa95da None admin] RBAC: Adding query filter params (role.id=a49254f8009c463e88e5e0b3909c5904, scope.project.id=0c5d68529f48460f9f6e03e3f8fb205b, user.id=0b74c769188e46608374f314768d312a) {{(pid=80540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 11:57:27.337223 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28336b92-1e51-4a19-b6a6-f9d7c5aa95da None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:57:27.339152 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28336b92-1e51-4a19-b6a6-f9d7c5aa95da None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:57:27.346460 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 129/251] 10.222.0.23 () {56 vars in 1374 bytes} [Thu May 19 11:57:27 2022] GET /identity/v3/role_assignments?user.id=0b74c769188e46608374f314768d312a&scope.project.id=0c5d68529f48460f9f6e03e3f8fb205b&role.id=a49254f8009c463e88e5e0b3909c5904 => generated 259 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 11:57:27.362064 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f36d530-cfc4-4cfb-9ca5-0598fe920a09 None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:57:27.404142 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963a94c8-e665-4ee8-9d91-2a954fa30369 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:57:27.405030 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963a94c8-e665-4ee8-9d91-2a954fa30369 None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:27.405317 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963a94c8-e665-4ee8-9d91-2a954fa30369 None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:27.405621 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963a94c8-e665-4ee8-9d91-2a954fa30369 None admin] PATH_INFO: `/v3/projects/0c5d68529f48460f9f6e03e3f8fb205b` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:27.406911 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963a94c8-e665-4ee8-9d91-2a954fa30369 None admin] RBAC: Authorizing `identity:get_project(project_id=0c5d68529f48460f9f6e03e3f8fb205b)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:57:27.408708 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963a94c8-e665-4ee8-9d91-2a954fa30369 None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:57:27.411079 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 123/252] 10.222.0.23 () {56 vars in 1186 bytes} [Thu May 19 11:57:27 2022] GET /identity/v3/projects/0c5d68529f48460f9f6e03e3f8fb205b => generated 302 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 11:57:27.417274 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28336b92-1e51-4a19-b6a6-f9d7c5aa95da None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:57:27.462666 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:57:27.572704 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28336b92-1e51-4a19-b6a6-f9d7c5aa95da None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:57:27.625943 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f664fb-1c78-4ac6-83be-88b51ed72db5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:57:27.626775 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f664fb-1c78-4ac6-83be-88b51ed72db5 None admin] REQUEST_METHOD: `PUT` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:27.627059 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f664fb-1c78-4ac6-83be-88b51ed72db5 None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:27.627338 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f664fb-1c78-4ac6-83be-88b51ed72db5 None admin] PATH_INFO: `/v3/projects/0c5d68529f48460f9f6e03e3f8fb205b/users/2d4dca52a2a64b5cb2ac901fb59ecdc2/roles/259eb400cdb4439db72d14456ae9cd08` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:27.633376 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f664fb-1c78-4ac6-83be-88b51ed72db5 None admin] RBAC: Authorizing `identity:create_grant(project_id=0c5d68529f48460f9f6e03e3f8fb205b, user_id=2d4dca52a2a64b5cb2ac901fb59ecdc2, role_id=259eb400cdb4439db72d14456ae9cd08)` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:57:27.634901 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f664fb-1c78-4ac6-83be-88b51ed72db5 None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:57:27.703961 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2d09a4-417b-4ace-a420-e8e07afad26e None None] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:27.704495 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2d09a4-417b-4ace-a420-e8e07afad26e None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:27.704495 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2d09a4-417b-4ace-a420-e8e07afad26e None None] PATH_INFO: `/` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:27.706063 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 124/253] 10.222.0.23 () {54 vars in 926 bytes} [Thu May 19 11:57:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 11:57:27.711244 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce43781-df42-4ae6-988f-a0c334417b61 None None] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:27.711534 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce43781-df42-4ae6-988f-a0c334417b61 None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:27.711840 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce43781-df42-4ae6-988f-a0c334417b61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:27.721226 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 130/254] 10.222.0.23 () {58 vars in 1439 bytes} [Thu May 19 11:57:27 2022] PUT /identity/v3/projects/0c5d68529f48460f9f6e03e3f8fb205b/users/2d4dca52a2a64b5cb2ac901fb59ecdc2/roles/259eb400cdb4439db72d14456ae9cd08 => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 11:57:27.741128 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.auth.core [None req-fce43781-df42-4ae6-988f-a0c334417b61 None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 11:57:27.742372 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-fce43781-df42-4ae6-988f-a0c334417b61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 11:57:27.806561 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7b1b67-c8a5-4855-9c6f-2ad564fe6238 None None] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:27.807221 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7b1b67-c8a5-4855-9c6f-2ad564fe6238 None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:27.807854 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7b1b67-c8a5-4855-9c6f-2ad564fe6238 None None] PATH_INFO: `/` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:27.810038 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 131/255] 10.222.0.23 () {54 vars in 926 bytes} [Thu May 19 11:57:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 11:57:27.817578 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863a3720-5f50-4fb7-a250-e5086229e170 None None] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:27.818247 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863a3720-5f50-4fb7-a250-e5086229e170 None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:27.818730 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863a3720-5f50-4fb7-a250-e5086229e170 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:27.822392 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:57:27.831867 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 125/256] 10.222.0.23 () {58 vars in 1023 bytes} [Thu May 19 11:57:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 11:57:27.842922 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.auth.core [None req-863a3720-5f50-4fb7-a250-e5086229e170 None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 11:57:27.844855 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-863a3720-5f50-4fb7-a250-e5086229e170 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 11:57:27.867759 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce43781-df42-4ae6-988f-a0c334417b61 None None] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:57:27.990565 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 132/257] 10.222.0.23 () {58 vars in 1023 bytes} [Thu May 19 11:57:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 11:57:28.043953 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863a3720-5f50-4fb7-a250-e5086229e170 None None] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:57:28.074950 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce43781-df42-4ae6-988f-a0c334417b61 None None] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:57:28.087433 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:57:28.114622 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff5be5c-383c-4003-b7e3-cabb9f027d82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:57:28.115675 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff5be5c-383c-4003-b7e3-cabb9f027d82 None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:28.116112 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff5be5c-383c-4003-b7e3-cabb9f027d82 None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:28.116488 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff5be5c-383c-4003-b7e3-cabb9f027d82 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:28.129740 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff5be5c-383c-4003-b7e3-cabb9f027d82 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 11:57:28.137992 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application [None req-5ff5be5c-383c-4003-b7e3-cabb9f027d82 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 11:57:28.137992 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 11:57:28.137992 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 11:57:28.137992 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 11:57:28.137992 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 11:57:28.137992 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 11:57:28.137992 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 11:57:28.137992 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 11:57:28.137992 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 11:57:28.137992 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 11:57:28.137992 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 11:57:28.137992 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 11:57:28.137992 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 11:57:28.137992 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 11:57:28.137992 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 11:57:28.137992 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 11:57:28.138951 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 11:57:28.138951 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 11:57:28.138951 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 11:57:28.138951 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 11:57:28.138951 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 11:57:28.138951 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 11:57:28.138951 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 11:57:28.138951 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 11:57:28.138951 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 11:57:28.138951 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.get_or_create( May 19 11:57:28.138951 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 11:57:28.138951 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application with Lock( May 19 11:57:28.138951 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 11:57:28.138951 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._enter() May 19 11:57:28.138951 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 11:57:28.138951 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 11:57:28.140655 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 11:57:28.140655 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.creator() May 19 11:57:28.140655 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 11:57:28.140655 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application created_value = creator( May 19 11:57:28.140655 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 11:57:28.140655 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 11:57:28.140655 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 11:57:28.140655 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 11:57:28.140655 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 11:57:28.140655 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 11:57:28.140655 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 11:57:28.140655 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application May 19 11:57:28.141256 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 126/258] 10.222.0.23 () {56 vars in 1096 bytes} [Thu May 19 11:57:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 11:57:28.149723 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff5be5c-383c-4003-b7e3-cabb9f027d82 None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:57:28.212056 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863a3720-5f50-4fb7-a250-e5086229e170 None None] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:57:28.282663 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:57:28.297357 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1543379c-6f33-475c-84b9-8a1939f0738d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:57:28.299317 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1543379c-6f33-475c-84b9-8a1939f0738d None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:28.299317 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1543379c-6f33-475c-84b9-8a1939f0738d None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:28.299317 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1543379c-6f33-475c-84b9-8a1939f0738d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:28.336827 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1543379c-6f33-475c-84b9-8a1939f0738d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 11:57:28.340719 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff5be5c-383c-4003-b7e3-cabb9f027d82 None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:57:28.353912 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application [None req-1543379c-6f33-475c-84b9-8a1939f0738d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 11:57:28.353912 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 11:57:28.353912 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 11:57:28.353912 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 11:57:28.353912 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 11:57:28.353912 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 11:57:28.353912 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 11:57:28.353912 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 11:57:28.353912 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 11:57:28.353912 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 11:57:28.353912 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 11:57:28.353912 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 11:57:28.353912 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 11:57:28.353912 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 11:57:28.353912 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 11:57:28.353912 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 11:57:28.354871 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 11:57:28.354871 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 11:57:28.354871 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 11:57:28.354871 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 11:57:28.354871 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 11:57:28.354871 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 11:57:28.354871 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 11:57:28.354871 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 11:57:28.354871 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 11:57:28.354871 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.get_or_create( May 19 11:57:28.354871 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 11:57:28.354871 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application with Lock( May 19 11:57:28.354871 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 11:57:28.354871 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._enter() May 19 11:57:28.354871 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 11:57:28.354871 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 11:57:28.355724 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 11:57:28.355724 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.creator() May 19 11:57:28.355724 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 11:57:28.355724 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application created_value = creator( May 19 11:57:28.355724 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 11:57:28.355724 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 11:57:28.355724 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 11:57:28.355724 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 11:57:28.355724 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 11:57:28.355724 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 11:57:28.355724 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 11:57:28.355724 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application May 19 11:57:28.355724 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 133/259] 10.222.0.23 () {56 vars in 1114 bytes} [Thu May 19 11:57:28 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 312 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 11:57:28.362379 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1543379c-6f33-475c-84b9-8a1939f0738d None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:57:28.387276 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e0f6fc-0b9e-444b-ba5d-e9ef66895027 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:57:28.388038 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e0f6fc-0b9e-444b-ba5d-e9ef66895027 None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:28.388280 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e0f6fc-0b9e-444b-ba5d-e9ef66895027 None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:28.388563 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e0f6fc-0b9e-444b-ba5d-e9ef66895027 None admin] PATH_INFO: `/v3/roles` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:28.389447 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e0f6fc-0b9e-444b-ba5d-e9ef66895027 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 11:57:28.389768 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e0f6fc-0b9e-444b-ba5d-e9ef66895027 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:57:28.391355 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e0f6fc-0b9e-444b-ba5d-e9ef66895027 None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:57:28.408323 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 127/260] 10.222.0.23 () {56 vars in 1099 bytes} [Thu May 19 11:57:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:57:28.414216 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e0f6fc-0b9e-444b-ba5d-e9ef66895027 None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:57:28.524354 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:57:28.537858 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1543379c-6f33-475c-84b9-8a1939f0738d None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:57:28.596513 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ede5081-6e97-43aa-86d7-5b39ba3f9d72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:57:28.597194 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ede5081-6e97-43aa-86d7-5b39ba3f9d72 None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:28.597481 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ede5081-6e97-43aa-86d7-5b39ba3f9d72 None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:28.597758 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ede5081-6e97-43aa-86d7-5b39ba3f9d72 None admin] PATH_INFO: `/v3/roles` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:28.598513 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ede5081-6e97-43aa-86d7-5b39ba3f9d72 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 11:57:28.598821 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ede5081-6e97-43aa-86d7-5b39ba3f9d72 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:57:28.600252 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ede5081-6e97-43aa-86d7-5b39ba3f9d72 None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:57:28.610041 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 134/261] 10.222.0.23 () {56 vars in 1111 bytes} [Thu May 19 11:57:28 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:57:28.615305 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ede5081-6e97-43aa-86d7-5b39ba3f9d72 None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:57:28.637418 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e0f6fc-0b9e-444b-ba5d-e9ef66895027 None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:57:28.676573 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:57:28.724550 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf883824-af9f-4762-ac1f-68e741ab453a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:57:28.725662 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf883824-af9f-4762-ac1f-68e741ab453a None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:28.725950 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf883824-af9f-4762-ac1f-68e741ab453a None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:28.726226 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf883824-af9f-4762-ac1f-68e741ab453a None admin] PATH_INFO: `/v3/users/4105c099a7824283aa03a6c3cc324850` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:28.727594 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf883824-af9f-4762-ac1f-68e741ab453a None admin] RBAC: Authorizing `identity:get_user(user_id=4105c099a7824283aa03a6c3cc324850)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:57:28.729140 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf883824-af9f-4762-ac1f-68e741ab453a None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:57:28.731247 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 128/262] 10.222.0.23 () {56 vars in 1177 bytes} [Thu May 19 11:57:28 2022] GET /identity/v3/users/4105c099a7824283aa03a6c3cc324850 => generated 283 bytes in 317 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 11:57:28.737573 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf883824-af9f-4762-ac1f-68e741ab453a None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:57:28.791932 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ede5081-6e97-43aa-86d7-5b39ba3f9d72 None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:57:28.849266 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85248de-00eb-426e-8686-c68570aefd8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:57:28.850540 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85248de-00eb-426e-8686-c68570aefd8c None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:28.851032 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85248de-00eb-426e-8686-c68570aefd8c None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:28.851531 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85248de-00eb-426e-8686-c68570aefd8c None admin] PATH_INFO: `/v3/users/4105c099a7824283aa03a6c3cc324850` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:28.854603 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85248de-00eb-426e-8686-c68570aefd8c None admin] RBAC: Authorizing `identity:get_user(user_id=4105c099a7824283aa03a6c3cc324850)` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:57:28.858513 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85248de-00eb-426e-8686-c68570aefd8c None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:57:28.871918 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 135/263] 10.222.0.23 () {56 vars in 1177 bytes} [Thu May 19 11:57:28 2022] GET /identity/v3/users/4105c099a7824283aa03a6c3cc324850 => generated 283 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:57:28.873339 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85248de-00eb-426e-8686-c68570aefd8c None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:57:28.896060 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:57:29.055635 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85248de-00eb-426e-8686-c68570aefd8c None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:57:29.059690 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf883824-af9f-4762-ac1f-68e741ab453a None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:57:29.129477 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37a56de-6ffb-4237-aaef-9a65eb33c4c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:57:29.130146 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37a56de-6ffb-4237-aaef-9a65eb33c4c1 None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:29.130431 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37a56de-6ffb-4237-aaef-9a65eb33c4c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:29.130703 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37a56de-6ffb-4237-aaef-9a65eb33c4c1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:29.134953 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54782373-dc64-4889-bc4c-3c7bbccaf571 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:57:29.135608 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54782373-dc64-4889-bc4c-3c7bbccaf571 None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:29.135928 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54782373-dc64-4889-bc4c-3c7bbccaf571 None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:29.136203 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54782373-dc64-4889-bc4c-3c7bbccaf571 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:29.263795 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54782373-dc64-4889-bc4c-3c7bbccaf571 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:57:29.266647 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54782373-dc64-4889-bc4c-3c7bbccaf571 None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:57:29.293153 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:57:29.295395 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:57:29.336984 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37a56de-6ffb-4237-aaef-9a65eb33c4c1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:57:29.338485 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37a56de-6ffb-4237-aaef-9a65eb33c4c1 None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:57:29.447295 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 136/264] 10.222.0.23 () {58 vars in 1282 bytes} [Thu May 19 11:57:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 574 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 11:57:29.453865 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54782373-dc64-4889-bc4c-3c7bbccaf571 None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:57:29.579778 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 129/265] 10.222.0.23 () {58 vars in 1282 bytes} [Thu May 19 11:57:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 843 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 19 11:57:29.586402 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37a56de-6ffb-4237-aaef-9a65eb33c4c1 None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:57:29.613255 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54782373-dc64-4889-bc4c-3c7bbccaf571 None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:57:29.627861 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:57:29.653432 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615d6033-070b-4d4d-b4d0-e1f8f986128e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:57:29.654455 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615d6033-070b-4d4d-b4d0-e1f8f986128e None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:29.654745 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615d6033-070b-4d4d-b4d0-e1f8f986128e None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:29.655023 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615d6033-070b-4d4d-b4d0-e1f8f986128e None admin] PATH_INFO: `/v3/projects/0c5d68529f48460f9f6e03e3f8fb205b` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:29.658176 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615d6033-070b-4d4d-b4d0-e1f8f986128e None admin] RBAC: Authorizing `identity:get_project(project_id=0c5d68529f48460f9f6e03e3f8fb205b)` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:57:29.659665 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615d6033-070b-4d4d-b4d0-e1f8f986128e None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:57:29.661775 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 137/266] 10.222.0.23 () {56 vars in 1186 bytes} [Thu May 19 11:57:29 2022] GET /identity/v3/projects/0c5d68529f48460f9f6e03e3f8fb205b => generated 302 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 11:57:29.669494 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615d6033-070b-4d4d-b4d0-e1f8f986128e None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:57:29.802433 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615d6033-070b-4d4d-b4d0-e1f8f986128e None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:57:29.816587 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:57:29.853995 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37a56de-6ffb-4237-aaef-9a65eb33c4c1 None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:57:29.867591 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7bc7308-e7d8-4242-ba4d-4f9813bb86ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:57:29.868503 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bc7308-e7d8-4242-ba4d-4f9813bb86ec None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:29.868972 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bc7308-e7d8-4242-ba4d-4f9813bb86ec None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:29.869401 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bc7308-e7d8-4242-ba4d-4f9813bb86ec None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:29.870388 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7bc7308-e7d8-4242-ba4d-4f9813bb86ec None admin] RBAC: Adding query filter params (role.id=8215583bb4224c75b093ae8e3993eb9f, scope.project.id=0c5d68529f48460f9f6e03e3f8fb205b, user.id=4105c099a7824283aa03a6c3cc324850) {{(pid=80540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 11:57:29.870854 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7bc7308-e7d8-4242-ba4d-4f9813bb86ec None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:57:29.872624 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7bc7308-e7d8-4242-ba4d-4f9813bb86ec None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:57:29.888729 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 138/267] 10.222.0.23 () {56 vars in 1374 bytes} [Thu May 19 11:57:29 2022] GET /identity/v3/role_assignments?user.id=4105c099a7824283aa03a6c3cc324850&scope.project.id=0c5d68529f48460f9f6e03e3f8fb205b&role.id=8215583bb4224c75b093ae8e3993eb9f => generated 608 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:57:29.956696 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5586a4ee-fec0-4aa8-84ce-d3273cc868b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:57:29.957594 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5586a4ee-fec0-4aa8-84ce-d3273cc868b0 None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:29.957880 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5586a4ee-fec0-4aa8-84ce-d3273cc868b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:29.958156 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5586a4ee-fec0-4aa8-84ce-d3273cc868b0 None admin] PATH_INFO: `/v3/projects/0c5d68529f48460f9f6e03e3f8fb205b` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:29.959467 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5586a4ee-fec0-4aa8-84ce-d3273cc868b0 None admin] RBAC: Authorizing `identity:get_project(project_id=0c5d68529f48460f9f6e03e3f8fb205b)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:57:29.960991 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5586a4ee-fec0-4aa8-84ce-d3273cc868b0 None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:57:29.962804 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 130/268] 10.222.0.23 () {56 vars in 1186 bytes} [Thu May 19 11:57:29 2022] GET /identity/v3/projects/0c5d68529f48460f9f6e03e3f8fb205b => generated 302 bytes in 377 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 11:57:29.969422 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7bc7308-e7d8-4242-ba4d-4f9813bb86ec None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:57:30.067425 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:57:30.140238 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7bc7308-e7d8-4242-ba4d-4f9813bb86ec None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:57:30.190729 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ce7cd1-5b28-4f6a-9504-83c90aeac912 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:57:30.191755 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ce7cd1-5b28-4f6a-9504-83c90aeac912 None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:30.192291 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ce7cd1-5b28-4f6a-9504-83c90aeac912 None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:30.192792 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ce7cd1-5b28-4f6a-9504-83c90aeac912 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:30.193840 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ce7cd1-5b28-4f6a-9504-83c90aeac912 None admin] RBAC: Adding query filter params (role.id=9db5671c65d3414ea2682afa2ccf1403, scope.project.id=0c5d68529f48460f9f6e03e3f8fb205b, user.id=4105c099a7824283aa03a6c3cc324850) {{(pid=80540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 11:57:30.194359 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ce7cd1-5b28-4f6a-9504-83c90aeac912 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:57:30.196336 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ce7cd1-5b28-4f6a-9504-83c90aeac912 None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:57:30.209524 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 139/269] 10.222.0.23 () {56 vars in 1374 bytes} [Thu May 19 11:57:29 2022] GET /identity/v3/role_assignments?user.id=4105c099a7824283aa03a6c3cc324850&scope.project.id=0c5d68529f48460f9f6e03e3f8fb205b&role.id=9db5671c65d3414ea2682afa2ccf1403 => generated 608 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:57:30.413507 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6745c85-af91-4f87-ad04-275e1c5ce875 None None] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:30.413691 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6745c85-af91-4f87-ad04-275e1c5ce875 None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:30.413977 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6745c85-af91-4f87-ad04-275e1c5ce875 None None] PATH_INFO: `/` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:30.415659 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 131/270] 10.222.0.23 () {54 vars in 926 bytes} [Thu May 19 11:57:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 11:57:30.421957 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c97f422-0b08-4f20-9417-da69b864e9d9 None None] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:30.422281 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c97f422-0b08-4f20-9417-da69b864e9d9 None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:30.422513 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c97f422-0b08-4f20-9417-da69b864e9d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:30.449030 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.auth.core [None req-9c97f422-0b08-4f20-9417-da69b864e9d9 None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 11:57:30.450262 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-9c97f422-0b08-4f20-9417-da69b864e9d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 11:57:30.513235 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8462814f-7395-4ce7-953a-ba993cc15199 None None] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:30.513561 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8462814f-7395-4ce7-953a-ba993cc15199 None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:30.513805 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8462814f-7395-4ce7-953a-ba993cc15199 None None] PATH_INFO: `/` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:30.516531 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 132/271] 10.222.0.23 () {54 vars in 926 bytes} [Thu May 19 11:57:30 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 11:57:30.523290 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e87eb0-b84c-49aa-9e26-68c437e50e61 None None] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:30.523603 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e87eb0-b84c-49aa-9e26-68c437e50e61 None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:30.523883 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e87eb0-b84c-49aa-9e26-68c437e50e61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:30.565126 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.auth.core [None req-a9e87eb0-b84c-49aa-9e26-68c437e50e61 None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 11:57:30.566850 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-a9e87eb0-b84c-49aa-9e26-68c437e50e61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 11:57:30.585335 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 140/272] 10.222.0.23 () {58 vars in 1023 bytes} [Thu May 19 11:57:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 11:57:30.621124 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c97f422-0b08-4f20-9417-da69b864e9d9 None None] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:57:30.675716 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:57:30.683949 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 133/273] 10.222.0.23 () {58 vars in 1023 bytes} [Thu May 19 11:57:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 11:57:30.708948 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e87eb0-b84c-49aa-9e26-68c437e50e61 None None] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:57:30.765020 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c97f422-0b08-4f20-9417-da69b864e9d9 None None] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:57:30.827209 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0087f128-02b7-4009-8c9c-49c0318a74bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:57:30.829755 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0087f128-02b7-4009-8c9c-49c0318a74bf None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:30.830287 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0087f128-02b7-4009-8c9c-49c0318a74bf None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:30.830853 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0087f128-02b7-4009-8c9c-49c0318a74bf None admin] PATH_INFO: `/v3/roles/member` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:30.844040 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:57:30.847616 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0087f128-02b7-4009-8c9c-49c0318a74bf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 11:57:30.858885 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application [None req-0087f128-02b7-4009-8c9c-49c0318a74bf None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 11:57:30.858885 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 11:57:30.858885 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 11:57:30.858885 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 11:57:30.858885 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 11:57:30.858885 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 11:57:30.858885 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 11:57:30.858885 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 11:57:30.858885 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 11:57:30.858885 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 11:57:30.858885 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 11:57:30.858885 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 11:57:30.858885 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 11:57:30.858885 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 11:57:30.858885 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 11:57:30.858885 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 11:57:30.859865 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 11:57:30.859865 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 11:57:30.859865 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 11:57:30.859865 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 11:57:30.859865 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 11:57:30.859865 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 11:57:30.859865 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 11:57:30.859865 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 11:57:30.859865 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 11:57:30.859865 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.get_or_create( May 19 11:57:30.859865 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 11:57:30.859865 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application with Lock( May 19 11:57:30.859865 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 11:57:30.859865 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._enter() May 19 11:57:30.859865 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 11:57:30.859865 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 11:57:30.860688 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 11:57:30.860688 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.creator() May 19 11:57:30.860688 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 11:57:30.860688 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application created_value = creator( May 19 11:57:30.860688 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 11:57:30.860688 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 11:57:30.860688 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 11:57:30.860688 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 11:57:30.860688 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 11:57:30.860688 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 11:57:30.860688 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 11:57:30.860688 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application May 19 11:57:30.860688 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 141/274] 10.222.0.23 () {56 vars in 1099 bytes} [Thu May 19 11:57:30 2022] GET /identity/v3/roles/member => generated 84 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 11:57:30.867100 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0087f128-02b7-4009-8c9c-49c0318a74bf None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:57:30.867262 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e87eb0-b84c-49aa-9e26-68c437e50e61 None None] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:57:30.910063 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642d73a4-8d42-43ab-acc6-ee44a8ed93a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:57:30.911145 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642d73a4-8d42-43ab-acc6-ee44a8ed93a1 None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:30.911520 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642d73a4-8d42-43ab-acc6-ee44a8ed93a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:30.911810 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642d73a4-8d42-43ab-acc6-ee44a8ed93a1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:30.913025 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642d73a4-8d42-43ab-acc6-ee44a8ed93a1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:57:30.914518 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642d73a4-8d42-43ab-acc6-ee44a8ed93a1 None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:57:30.923969 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 134/275] 10.222.0.23 () {56 vars in 1108 bytes} [Thu May 19 11:57:30 2022] GET /identity/v3/domains/default => generated 199 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 11:57:30.923969 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3981355-079d-494a-a6ce-4ea84ff60d5b None None] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:30.923969 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3981355-079d-494a-a6ce-4ea84ff60d5b None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:30.923969 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3981355-079d-494a-a6ce-4ea84ff60d5b None None] PATH_INFO: `/` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:30.923969 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 135/276] 10.222.0.23 () {54 vars in 926 bytes} [Thu May 19 11:57:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 11:57:30.924581 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3981355-079d-494a-a6ce-4ea84ff60d5b None None] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:57:30.990606 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:57:30.995037 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0087f128-02b7-4009-8c9c-49c0318a74bf None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:57:31.034781 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9530291c-63d2-43db-83da-ee65506ba038 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:57:31.035462 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9530291c-63d2-43db-83da-ee65506ba038 None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:31.035751 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9530291c-63d2-43db-83da-ee65506ba038 None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:31.036146 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9530291c-63d2-43db-83da-ee65506ba038 None admin] PATH_INFO: `/v3/roles` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:31.036827 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9530291c-63d2-43db-83da-ee65506ba038 None admin] RBAC: Adding query filter params (name=member) {{(pid=80540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 11:57:31.037655 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9530291c-63d2-43db-83da-ee65506ba038 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:57:31.038559 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9530291c-63d2-43db-83da-ee65506ba038 None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:57:31.047231 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 142/277] 10.222.0.23 () {56 vars in 1101 bytes} [Thu May 19 11:57:30 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 11:57:31.048403 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04e2921-8355-4a47-9cf8-7b9565058a9e None None] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:31.048696 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04e2921-8355-4a47-9cf8-7b9565058a9e None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:31.048982 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04e2921-8355-4a47-9cf8-7b9565058a9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:31.055830 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3981355-079d-494a-a6ce-4ea84ff60d5b None None] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:57:31.066123 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.auth.core [None req-b04e2921-8355-4a47-9cf8-7b9565058a9e None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 11:57:31.067330 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-b04e2921-8355-4a47-9cf8-7b9565058a9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 11:57:31.092463 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b94afe-35c2-4595-8908-13c6324e27f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:57:31.093213 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b94afe-35c2-4595-8908-13c6324e27f4 None admin] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:31.093434 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b94afe-35c2-4595-8908-13c6324e27f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:31.093700 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b94afe-35c2-4595-8908-13c6324e27f4 None admin] PATH_INFO: `/v3/users` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:31.094550 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b94afe-35c2-4595-8908-13c6324e27f4 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:57:31.096033 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b94afe-35c2-4595-8908-13c6324e27f4 None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:57:31.138117 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 143/278] 10.222.0.23 () {58 vars in 1023 bytes} [Thu May 19 11:57:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 11:57:31.139457 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbea976-d73c-40cc-b09b-7260c6ad5fe0 None None] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:31.139752 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbea976-d73c-40cc-b09b-7260c6ad5fe0 None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:31.140710 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbea976-d73c-40cc-b09b-7260c6ad5fe0 None None] PATH_INFO: `/` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:31.142216 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 144/279] 10.222.0.23 () {54 vars in 926 bytes} [Thu May 19 11:57:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 11:57:31.142932 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbea976-d73c-40cc-b09b-7260c6ad5fe0 None None] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:57:31.206015 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 136/280] 10.222.0.23 () {60 vars in 1132 bytes} [Thu May 19 11:57:30 2022] POST /identity/v3/users => generated 293 bytes in 283 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 11:57:31.217445 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1f748a-d3ba-4fdc-aaca-8d8b2f98b7ce None None] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:31.217769 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1f748a-d3ba-4fdc-aaca-8d8b2f98b7ce None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:31.218674 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1f748a-d3ba-4fdc-aaca-8d8b2f98b7ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:31.244827 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.auth.core [None req-9a1f748a-d3ba-4fdc-aaca-8d8b2f98b7ce None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 11:57:31.246730 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-9a1f748a-d3ba-4fdc-aaca-8d8b2f98b7ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 11:57:31.250176 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:57:31.257307 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:57:31.269514 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbea976-d73c-40cc-b09b-7260c6ad5fe0 None None] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:57:31.313368 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1da04a6-524d-470b-aeb6-29bf1d5f6842 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:57:31.314783 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1da04a6-524d-470b-aeb6-29bf1d5f6842 None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:31.315090 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1da04a6-524d-470b-aeb6-29bf1d5f6842 None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:31.315358 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1da04a6-524d-470b-aeb6-29bf1d5f6842 None admin] PATH_INFO: `/v3/users/2f54e6b77e7f4ffd8c0a2a2105855fae` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:31.335394 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1da04a6-524d-470b-aeb6-29bf1d5f6842 None admin] RBAC: Authorizing `identity:get_user(user_id=2f54e6b77e7f4ffd8c0a2a2105855fae)` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:57:31.335668 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 137/281] 10.222.0.23 () {58 vars in 1023 bytes} [Thu May 19 11:57:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 11:57:31.336818 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1f748a-d3ba-4fdc-aaca-8d8b2f98b7ce None None] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:57:31.336917 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1da04a6-524d-470b-aeb6-29bf1d5f6842 None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:57:31.343884 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 145/282] 10.222.0.23 () {56 vars in 1177 bytes} [Thu May 19 11:57:31 2022] GET /identity/v3/users/2f54e6b77e7f4ffd8c0a2a2105855fae => generated 293 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:57:31.347509 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1da04a6-524d-470b-aeb6-29bf1d5f6842 None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:57:31.486551 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1da04a6-524d-470b-aeb6-29bf1d5f6842 None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:57:31.487025 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1f748a-d3ba-4fdc-aaca-8d8b2f98b7ce None None] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:57:31.537610 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c8d6ce-a396-4747-a6b9-9be0271d54a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:57:31.538421 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c8d6ce-a396-4747-a6b9-9be0271d54a9 None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:31.538696 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c8d6ce-a396-4747-a6b9-9be0271d54a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:31.538973 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c8d6ce-a396-4747-a6b9-9be0271d54a9 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:31.543224 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7670da87-042d-4b20-9048-fc0db345dadc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:57:31.543933 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7670da87-042d-4b20-9048-fc0db345dadc None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:31.544224 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7670da87-042d-4b20-9048-fc0db345dadc None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:31.544564 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7670da87-042d-4b20-9048-fc0db345dadc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:31.545409 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7670da87-042d-4b20-9048-fc0db345dadc None admin] RBAC: Adding query filter params (role.id=259eb400cdb4439db72d14456ae9cd08, user.id=2f54e6b77e7f4ffd8c0a2a2105855fae, scope.system=all) {{(pid=80540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 11:57:31.545726 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7670da87-042d-4b20-9048-fc0db345dadc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:57:31.547772 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7670da87-042d-4b20-9048-fc0db345dadc None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:57:31.551436 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:57:31.555095 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c8d6ce-a396-4747-a6b9-9be0271d54a9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 11:57:31.557081 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 146/283] 10.222.0.23 () {56 vars in 1308 bytes} [Thu May 19 11:57:31 2022] GET /identity/v3/role_assignments?user.id=2f54e6b77e7f4ffd8c0a2a2105855fae&scope.system=all&role.id=259eb400cdb4439db72d14456ae9cd08 => generated 226 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:57:31.557917 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7670da87-042d-4b20-9048-fc0db345dadc None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:57:31.563290 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application [None req-b6c8d6ce-a396-4747-a6b9-9be0271d54a9 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 19 11:57:31.563290 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 11:57:31.563290 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 11:57:31.563290 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 11:57:31.563290 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 11:57:31.563290 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 11:57:31.563290 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 11:57:31.563290 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 11:57:31.563290 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 11:57:31.563290 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 11:57:31.563290 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 11:57:31.563290 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 11:57:31.563290 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 11:57:31.563290 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 11:57:31.563290 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 11:57:31.563290 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 11:57:31.565529 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 11:57:31.565529 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 11:57:31.565529 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 11:57:31.565529 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 11:57:31.565529 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 11:57:31.565529 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 11:57:31.565529 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 11:57:31.565529 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 11:57:31.565529 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 11:57:31.565529 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.get_or_create( May 19 11:57:31.565529 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 11:57:31.565529 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application with Lock( May 19 11:57:31.565529 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 11:57:31.565529 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._enter() May 19 11:57:31.565529 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 11:57:31.565529 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 11:57:31.566340 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 11:57:31.566340 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.creator() May 19 11:57:31.566340 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 11:57:31.566340 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application created_value = creator( May 19 11:57:31.566340 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 11:57:31.566340 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 11:57:31.566340 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 11:57:31.566340 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 11:57:31.566340 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 11:57:31.566340 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 11:57:31.566340 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 19 11:57:31.566340 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application May 19 11:57:31.566340 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 138/284] 10.222.0.23 () {56 vars in 1099 bytes} [Thu May 19 11:57:31 2022] GET /identity/v3/roles/reader => generated 84 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 11:57:31.570792 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c8d6ce-a396-4747-a6b9-9be0271d54a9 None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:57:31.583952 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:57:31.709437 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7670da87-042d-4b20-9048-fc0db345dadc None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:57:31.722449 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c8d6ce-a396-4747-a6b9-9be0271d54a9 None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:57:31.771917 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82951766-e161-447b-a0e4-52b5276255b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:57:31.771917 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82951766-e161-447b-a0e4-52b5276255b3 None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:31.771917 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82951766-e161-447b-a0e4-52b5276255b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:31.771917 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82951766-e161-447b-a0e4-52b5276255b3 None admin] PATH_INFO: `/v3/roles` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:31.772294 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82951766-e161-447b-a0e4-52b5276255b3 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 11:57:31.772294 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82951766-e161-447b-a0e4-52b5276255b3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:57:31.772294 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82951766-e161-447b-a0e4-52b5276255b3 None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:57:31.773466 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395a3116-7d11-49c2-a836-dd6a54f7fddf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:57:31.774512 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395a3116-7d11-49c2-a836-dd6a54f7fddf None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:31.774997 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395a3116-7d11-49c2-a836-dd6a54f7fddf None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:31.775483 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395a3116-7d11-49c2-a836-dd6a54f7fddf None admin] PATH_INFO: `/v3/roles/member` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:31.787816 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 139/285] 10.222.0.23 () {56 vars in 1101 bytes} [Thu May 19 11:57:31 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 11:57:31.795246 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395a3116-7d11-49c2-a836-dd6a54f7fddf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 11:57:31.805236 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82951766-e161-447b-a0e4-52b5276255b3 None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:57:31.813096 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application [None req-395a3116-7d11-49c2-a836-dd6a54f7fddf None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 11:57:31.813096 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 11:57:31.813096 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 11:57:31.813096 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 11:57:31.813096 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 11:57:31.813096 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 11:57:31.813096 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 11:57:31.813096 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 11:57:31.813096 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 11:57:31.813096 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 11:57:31.813096 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 11:57:31.813096 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 11:57:31.813096 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 11:57:31.813096 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 11:57:31.813096 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 11:57:31.813096 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 11:57:31.813996 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 11:57:31.813996 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 11:57:31.813996 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 11:57:31.813996 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 11:57:31.813996 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 11:57:31.813996 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 11:57:31.813996 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 11:57:31.813996 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 11:57:31.813996 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 11:57:31.813996 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.get_or_create( May 19 11:57:31.813996 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 11:57:31.813996 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application with Lock( May 19 11:57:31.813996 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 11:57:31.813996 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._enter() May 19 11:57:31.813996 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 11:57:31.813996 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 11:57:31.814767 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 11:57:31.814767 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.creator() May 19 11:57:31.814767 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 11:57:31.814767 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application created_value = creator( May 19 11:57:31.814767 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 11:57:31.814767 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 11:57:31.814767 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 11:57:31.814767 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 11:57:31.814767 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 11:57:31.814767 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 11:57:31.814767 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 11:57:31.814767 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application May 19 11:57:31.817849 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 147/286] 10.222.0.23 () {56 vars in 1099 bytes} [Thu May 19 11:57:31 2022] GET /identity/v3/roles/member => generated 84 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 11:57:31.823999 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395a3116-7d11-49c2-a836-dd6a54f7fddf None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:57:31.879579 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:57:31.925102 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82951766-e161-447b-a0e4-52b5276255b3 None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:57:31.973371 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aeee4dc-7c20-4265-aeb5-e257a51e690e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:57:31.974246 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aeee4dc-7c20-4265-aeb5-e257a51e690e None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:31.974527 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aeee4dc-7c20-4265-aeb5-e257a51e690e None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:31.974798 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aeee4dc-7c20-4265-aeb5-e257a51e690e None admin] PATH_INFO: `/v3/users/0b74c769188e46608374f314768d312a` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:31.979228 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aeee4dc-7c20-4265-aeb5-e257a51e690e None admin] RBAC: Authorizing `identity:get_user(user_id=0b74c769188e46608374f314768d312a)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:57:31.980715 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aeee4dc-7c20-4265-aeb5-e257a51e690e None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:57:31.982377 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 140/287] 10.222.0.23 () {56 vars in 1177 bytes} [Thu May 19 11:57:31 2022] GET /identity/v3/users/0b74c769188e46608374f314768d312a => generated 297 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 11:57:31.983876 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:57:31.988764 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aeee4dc-7c20-4265-aeb5-e257a51e690e None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:57:32.062634 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395a3116-7d11-49c2-a836-dd6a54f7fddf None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:57:32.105307 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aeee4dc-7c20-4265-aeb5-e257a51e690e None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:57:32.141589 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c105c6-34ed-401c-80ff-419810e051ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:57:32.142460 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c105c6-34ed-401c-80ff-419810e051ae None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:32.142739 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c105c6-34ed-401c-80ff-419810e051ae None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:32.143010 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c105c6-34ed-401c-80ff-419810e051ae None admin] PATH_INFO: `/v3/projects/0c5d68529f48460f9f6e03e3f8fb205b` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:32.144446 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c105c6-34ed-401c-80ff-419810e051ae None admin] RBAC: Authorizing `identity:get_project(project_id=0c5d68529f48460f9f6e03e3f8fb205b)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:57:32.146042 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c105c6-34ed-401c-80ff-419810e051ae None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:57:32.148198 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:57:32.148497 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 141/288] 10.222.0.23 () {56 vars in 1186 bytes} [Thu May 19 11:57:31 2022] GET /identity/v3/projects/0c5d68529f48460f9f6e03e3f8fb205b => generated 302 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:57:32.153970 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c105c6-34ed-401c-80ff-419810e051ae None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:57:32.174972 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b30abd7-2d5f-498a-900e-d55a6504d3d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:57:32.175660 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b30abd7-2d5f-498a-900e-d55a6504d3d6 None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:32.175985 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b30abd7-2d5f-498a-900e-d55a6504d3d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:32.176262 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b30abd7-2d5f-498a-900e-d55a6504d3d6 None admin] PATH_INFO: `/v3/roles` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:32.177016 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b30abd7-2d5f-498a-900e-d55a6504d3d6 None admin] RBAC: Adding query filter params (name=member) {{(pid=80540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 11:57:32.177323 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b30abd7-2d5f-498a-900e-d55a6504d3d6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:57:32.178725 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b30abd7-2d5f-498a-900e-d55a6504d3d6 None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:57:32.191214 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 148/289] 10.222.0.23 () {56 vars in 1101 bytes} [Thu May 19 11:57:31 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 367 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:57:32.196752 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b30abd7-2d5f-498a-900e-d55a6504d3d6 None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:57:32.323763 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b30abd7-2d5f-498a-900e-d55a6504d3d6 None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:57:32.359876 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c105c6-34ed-401c-80ff-419810e051ae None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:57:32.363871 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3899ac9-1afe-4825-a6e6-622174e82e52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:57:32.364791 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3899ac9-1afe-4825-a6e6-622174e82e52 None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:32.365078 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3899ac9-1afe-4825-a6e6-622174e82e52 None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:32.365354 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3899ac9-1afe-4825-a6e6-622174e82e52 None admin] PATH_INFO: `/v3/users/2d4dca52a2a64b5cb2ac901fb59ecdc2` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:32.366560 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3899ac9-1afe-4825-a6e6-622174e82e52 None admin] RBAC: Authorizing `identity:get_user(user_id=2d4dca52a2a64b5cb2ac901fb59ecdc2)` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:57:32.368050 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3899ac9-1afe-4825-a6e6-622174e82e52 None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:57:32.375623 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 149/290] 10.222.0.23 () {56 vars in 1177 bytes} [Thu May 19 11:57:32 2022] GET /identity/v3/users/2d4dca52a2a64b5cb2ac901fb59ecdc2 => generated 297 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 11:57:32.376951 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3899ac9-1afe-4825-a6e6-622174e82e52 None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:57:32.383840 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:57:32.442463 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a49beba-d8e7-40e6-93a9-5cde8253b04d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:57:32.443370 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a49beba-d8e7-40e6-93a9-5cde8253b04d None admin] REQUEST_METHOD: `PUT` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:32.443650 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a49beba-d8e7-40e6-93a9-5cde8253b04d None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:32.443947 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a49beba-d8e7-40e6-93a9-5cde8253b04d None admin] PATH_INFO: `/v3/projects/0c5d68529f48460f9f6e03e3f8fb205b/users/0b74c769188e46608374f314768d312a/roles/a49254f8009c463e88e5e0b3909c5904` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:32.447460 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a49beba-d8e7-40e6-93a9-5cde8253b04d None admin] RBAC: Authorizing `identity:create_grant(project_id=0c5d68529f48460f9f6e03e3f8fb205b, user_id=0b74c769188e46608374f314768d312a, role_id=a49254f8009c463e88e5e0b3909c5904)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:57:32.450631 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a49beba-d8e7-40e6-93a9-5cde8253b04d None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:57:32.506832 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3899ac9-1afe-4825-a6e6-622174e82e52 None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:57:32.548751 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 142/291] 10.222.0.23 () {58 vars in 1439 bytes} [Thu May 19 11:57:32 2022] PUT /identity/v3/projects/0c5d68529f48460f9f6e03e3f8fb205b/users/0b74c769188e46608374f314768d312a/roles/a49254f8009c463e88e5e0b3909c5904 => generated 0 bytes in 395 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 11:57:32.553762 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254a85d4-64dd-4e6d-be59-ce6fefe0ffd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:57:32.554576 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254a85d4-64dd-4e6d-be59-ce6fefe0ffd0 None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:32.554950 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254a85d4-64dd-4e6d-be59-ce6fefe0ffd0 None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:32.555124 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254a85d4-64dd-4e6d-be59-ce6fefe0ffd0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:32.663178 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254a85d4-64dd-4e6d-be59-ce6fefe0ffd0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:57:32.664754 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254a85d4-64dd-4e6d-be59-ce6fefe0ffd0 None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:57:32.666996 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:57:32.790866 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 150/292] 10.222.0.23 () {58 vars in 1282 bytes} [Thu May 19 11:57:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 414 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 11:57:32.796993 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a49beba-d8e7-40e6-93a9-5cde8253b04d None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:57:32.887120 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:57:32.949432 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a49beba-d8e7-40e6-93a9-5cde8253b04d None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:57:33.001537 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7524e209-0463-4470-ac5e-9e960842f910 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:57:33.002472 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7524e209-0463-4470-ac5e-9e960842f910 None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:33.002762 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7524e209-0463-4470-ac5e-9e960842f910 None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:33.003041 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7524e209-0463-4470-ac5e-9e960842f910 None admin] PATH_INFO: `/v3/projects/0c5d68529f48460f9f6e03e3f8fb205b` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:33.004294 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7524e209-0463-4470-ac5e-9e960842f910 None admin] RBAC: Authorizing `identity:get_project(project_id=0c5d68529f48460f9f6e03e3f8fb205b)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:57:33.005759 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7524e209-0463-4470-ac5e-9e960842f910 None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:57:33.007809 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 143/293] 10.222.0.23 () {56 vars in 1186 bytes} [Thu May 19 11:57:32 2022] GET /identity/v3/projects/0c5d68529f48460f9f6e03e3f8fb205b => generated 302 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:57:33.013365 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254a85d4-64dd-4e6d-be59-ce6fefe0ffd0 None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:57:33.143961 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254a85d4-64dd-4e6d-be59-ce6fefe0ffd0 None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:57:33.148127 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:57:33.183273 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135ba3a7-6c49-486a-b60c-7ce86389b18b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:57:33.183965 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135ba3a7-6c49-486a-b60c-7ce86389b18b None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:33.184252 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135ba3a7-6c49-486a-b60c-7ce86389b18b None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:33.184527 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135ba3a7-6c49-486a-b60c-7ce86389b18b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:33.185354 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-135ba3a7-6c49-486a-b60c-7ce86389b18b None admin] RBAC: Adding query filter params (role.id=259eb400cdb4439db72d14456ae9cd08, scope.project.id=0c5d68529f48460f9f6e03e3f8fb205b, user.id=2d4dca52a2a64b5cb2ac901fb59ecdc2) {{(pid=80540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 11:57:33.185663 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-135ba3a7-6c49-486a-b60c-7ce86389b18b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:57:33.187638 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-135ba3a7-6c49-486a-b60c-7ce86389b18b None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:57:33.194301 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 151/294] 10.222.0.23 () {56 vars in 1374 bytes} [Thu May 19 11:57:33 2022] GET /identity/v3/role_assignments?user.id=2d4dca52a2a64b5cb2ac901fb59ecdc2&scope.project.id=0c5d68529f48460f9f6e03e3f8fb205b&role.id=259eb400cdb4439db72d14456ae9cd08 => generated 608 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:57:34.006114 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55345ed0-cf59-45a2-8774-b8e25984faff None None] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:34.006660 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55345ed0-cf59-45a2-8774-b8e25984faff None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:34.006660 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55345ed0-cf59-45a2-8774-b8e25984faff None None] PATH_INFO: `/` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:34.008512 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 144/295] 10.222.0.23 () {54 vars in 926 bytes} [Thu May 19 11:57:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 11:57:34.014381 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f555061-5621-4031-af3e-acb0ee04f301 None None] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:34.015557 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f555061-5621-4031-af3e-acb0ee04f301 None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:34.015817 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f555061-5621-4031-af3e-acb0ee04f301 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:34.035101 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.auth.core [None req-0f555061-5621-4031-af3e-acb0ee04f301 None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 11:57:34.036374 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-0f555061-5621-4031-af3e-acb0ee04f301 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 11:57:34.112822 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 152/296] 10.222.0.23 () {58 vars in 1023 bytes} [Thu May 19 11:57:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 11:57:34.136284 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55345ed0-cf59-45a2-8774-b8e25984faff None None] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:57:34.208600 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cd98b1-7e8d-4c9e-a426-2d062545ef52 None None] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:34.208902 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cd98b1-7e8d-4c9e-a426-2d062545ef52 None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:34.209124 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cd98b1-7e8d-4c9e-a426-2d062545ef52 None None] PATH_INFO: `/` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:34.210740 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 153/297] 10.222.0.23 () {54 vars in 926 bytes} [Thu May 19 11:57:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 11:57:34.215654 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cc4dd2-2d85-460b-8705-f9dd2880b55c None None] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:34.215968 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cc4dd2-2d85-460b-8705-f9dd2880b55c None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:34.216246 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cc4dd2-2d85-460b-8705-f9dd2880b55c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:34.233551 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.auth.core [None req-66cc4dd2-2d85-460b-8705-f9dd2880b55c None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 11:57:34.234956 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-66cc4dd2-2d85-460b-8705-f9dd2880b55c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 11:57:34.250805 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55345ed0-cf59-45a2-8774-b8e25984faff None None] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:57:34.284276 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:57:34.292295 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419c0773-728e-40aa-bf4d-24ef4b4bb2c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:57:34.293101 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419c0773-728e-40aa-bf4d-24ef4b4bb2c6 None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:34.293387 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419c0773-728e-40aa-bf4d-24ef4b4bb2c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:34.293780 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419c0773-728e-40aa-bf4d-24ef4b4bb2c6 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:34.305810 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 154/298] 10.222.0.23 () {58 vars in 1023 bytes} [Thu May 19 11:57:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 11:57:34.310636 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419c0773-728e-40aa-bf4d-24ef4b4bb2c6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 11:57:34.323385 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application [None req-419c0773-728e-40aa-bf4d-24ef4b4bb2c6 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 19 11:57:34.323385 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 11:57:34.323385 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 11:57:34.323385 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 11:57:34.323385 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 11:57:34.323385 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 11:57:34.323385 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 11:57:34.323385 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 11:57:34.323385 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 11:57:34.323385 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 11:57:34.323385 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 11:57:34.323385 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 11:57:34.323385 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 11:57:34.323385 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 11:57:34.323385 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 11:57:34.323385 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 11:57:34.324406 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 11:57:34.324406 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 11:57:34.324406 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 11:57:34.324406 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 11:57:34.324406 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 11:57:34.324406 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 11:57:34.324406 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 11:57:34.324406 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 11:57:34.324406 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 11:57:34.324406 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.get_or_create( May 19 11:57:34.324406 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 11:57:34.324406 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application with Lock( May 19 11:57:34.324406 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 11:57:34.324406 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._enter() May 19 11:57:34.324406 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 11:57:34.324406 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 11:57:34.325579 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 11:57:34.325579 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.creator() May 19 11:57:34.325579 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 11:57:34.325579 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application created_value = creator( May 19 11:57:34.325579 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 11:57:34.325579 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 11:57:34.325579 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 11:57:34.325579 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 11:57:34.325579 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 11:57:34.325579 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 11:57:34.325579 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 19 11:57:34.325579 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application May 19 11:57:34.325579 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 145/299] 10.222.0.23 () {56 vars in 1099 bytes} [Thu May 19 11:57:34 2022] GET /identity/v3/roles/reader => generated 84 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 11:57:34.329941 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66cc4dd2-2d85-460b-8705-f9dd2880b55c None None] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:57:34.331982 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419c0773-728e-40aa-bf4d-24ef4b4bb2c6 None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:57:34.358356 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:57:34.433292 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66cc4dd2-2d85-460b-8705-f9dd2880b55c None None] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:57:34.446322 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419c0773-728e-40aa-bf4d-24ef4b4bb2c6 None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:57:34.469250 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310ac0b1-fe45-420b-a662-2faa8d8b8c3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:57:34.469962 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310ac0b1-fe45-420b-a662-2faa8d8b8c3f None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:34.470248 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310ac0b1-fe45-420b-a662-2faa8d8b8c3f None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:34.470521 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310ac0b1-fe45-420b-a662-2faa8d8b8c3f None admin] PATH_INFO: `/v3/domains/default` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:34.471684 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310ac0b1-fe45-420b-a662-2faa8d8b8c3f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:57:34.473390 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310ac0b1-fe45-420b-a662-2faa8d8b8c3f None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:57:34.475046 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 155/300] 10.222.0.23 () {56 vars in 1108 bytes} [Thu May 19 11:57:34 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:57:34.477327 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee945bde-27f4-49ca-98b8-f2a7d2608b56 None None] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:34.477619 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee945bde-27f4-49ca-98b8-f2a7d2608b56 None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:34.477901 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee945bde-27f4-49ca-98b8-f2a7d2608b56 None None] PATH_INFO: `/` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:34.479286 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 156/301] 10.222.0.23 () {54 vars in 926 bytes} [Thu May 19 11:57:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 11:57:34.480557 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee945bde-27f4-49ca-98b8-f2a7d2608b56 None None] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:57:34.490225 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fdf8ed-b88e-4d9e-b67a-d56bbe4ae5f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:57:34.491240 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fdf8ed-b88e-4d9e-b67a-d56bbe4ae5f7 None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:34.491715 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fdf8ed-b88e-4d9e-b67a-d56bbe4ae5f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:34.492209 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fdf8ed-b88e-4d9e-b67a-d56bbe4ae5f7 None admin] PATH_INFO: `/v3/roles` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:34.493258 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41fdf8ed-b88e-4d9e-b67a-d56bbe4ae5f7 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 11:57:34.493749 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41fdf8ed-b88e-4d9e-b67a-d56bbe4ae5f7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:57:34.495501 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41fdf8ed-b88e-4d9e-b67a-d56bbe4ae5f7 None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:57:34.512965 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 146/302] 10.222.0.23 () {56 vars in 1101 bytes} [Thu May 19 11:57:34 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 11:57:34.513666 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c420fced-b069-4de1-a86d-b4b3018cce53 None None] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:34.513959 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c420fced-b069-4de1-a86d-b4b3018cce53 None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:34.514235 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c420fced-b069-4de1-a86d-b4b3018cce53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:34.532374 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.auth.core [None req-c420fced-b069-4de1-a86d-b4b3018cce53 None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 11:57:34.533555 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-c420fced-b069-4de1-a86d-b4b3018cce53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 11:57:34.581104 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:57:34.595960 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:57:34.603329 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee945bde-27f4-49ca-98b8-f2a7d2608b56 None None] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:57:34.606533 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 147/303] 10.222.0.23 () {58 vars in 1023 bytes} [Thu May 19 11:57:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 11:57:34.607236 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c420fced-b069-4de1-a86d-b4b3018cce53 None None] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:57:34.653644 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea5a6cc-0eb2-4312-9602-a850b662d9e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:57:34.654381 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea5a6cc-0eb2-4312-9602-a850b662d9e5 None admin] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:34.654600 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea5a6cc-0eb2-4312-9602-a850b662d9e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:34.654863 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea5a6cc-0eb2-4312-9602-a850b662d9e5 None admin] PATH_INFO: `/v3/groups` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:34.656093 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea5a6cc-0eb2-4312-9602-a850b662d9e5 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:57:34.658581 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea5a6cc-0eb2-4312-9602-a850b662d9e5 None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:57:34.716982 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c420fced-b069-4de1-a86d-b4b3018cce53 None None] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:57:34.727715 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 157/304] 10.222.0.23 () {60 vars in 1134 bytes} [Thu May 19 11:57:34 2022] POST /identity/v3/groups => generated 229 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 11:57:34.728641 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea5a6cc-0eb2-4312-9602-a850b662d9e5 None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:57:34.783582 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547cdb4a-27fa-44c2-b7fb-ad014dd6ec04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:57:34.785172 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547cdb4a-27fa-44c2-b7fb-ad014dd6ec04 None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:34.785657 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547cdb4a-27fa-44c2-b7fb-ad014dd6ec04 None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:34.786404 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547cdb4a-27fa-44c2-b7fb-ad014dd6ec04 None admin] PATH_INFO: `/v3/users/07e60193e21e42f4bd442f0292b93267` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:34.804666 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547cdb4a-27fa-44c2-b7fb-ad014dd6ec04 None admin] RBAC: Authorizing `identity:get_user(user_id=07e60193e21e42f4bd442f0292b93267)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:57:34.806865 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547cdb4a-27fa-44c2-b7fb-ad014dd6ec04 None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:57:34.809259 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 148/305] 10.222.0.23 () {56 vars in 1177 bytes} [Thu May 19 11:57:34 2022] GET /identity/v3/users/07e60193e21e42f4bd442f0292b93267 => generated 293 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:57:34.815960 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547cdb4a-27fa-44c2-b7fb-ad014dd6ec04 None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:57:34.858127 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea5a6cc-0eb2-4312-9602-a850b662d9e5 None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:57:34.866103 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:57:34.911291 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a25955-0bb6-45e5-aafc-7a37bb7b64de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:57:34.912491 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a25955-0bb6-45e5-aafc-7a37bb7b64de None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:34.913024 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a25955-0bb6-45e5-aafc-7a37bb7b64de None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:34.913490 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a25955-0bb6-45e5-aafc-7a37bb7b64de None admin] PATH_INFO: `/v3/roles/member` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:34.928092 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a25955-0bb6-45e5-aafc-7a37bb7b64de None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 11:57:34.937623 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application [None req-39a25955-0bb6-45e5-aafc-7a37bb7b64de None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 11:57:34.937623 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 11:57:34.937623 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 11:57:34.937623 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 11:57:34.937623 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 11:57:34.937623 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 11:57:34.937623 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 11:57:34.937623 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 11:57:34.937623 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 11:57:34.937623 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 11:57:34.937623 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 11:57:34.937623 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 11:57:34.937623 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 11:57:34.937623 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 11:57:34.937623 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 11:57:34.937623 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 11:57:34.938565 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 11:57:34.938565 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 11:57:34.938565 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 11:57:34.938565 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 11:57:34.938565 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 11:57:34.938565 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 11:57:34.938565 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 11:57:34.938565 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 11:57:34.938565 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 11:57:34.938565 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.get_or_create( May 19 11:57:34.938565 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 11:57:34.938565 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application with Lock( May 19 11:57:34.938565 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 11:57:34.938565 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._enter() May 19 11:57:34.938565 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 11:57:34.938565 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 11:57:34.939349 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 11:57:34.939349 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.creator() May 19 11:57:34.939349 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 11:57:34.939349 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application created_value = creator( May 19 11:57:34.939349 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 11:57:34.939349 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 11:57:34.939349 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 11:57:34.939349 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 11:57:34.939349 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 11:57:34.939349 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 11:57:34.939349 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 11:57:34.939349 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application May 19 11:57:34.942445 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 158/306] 10.222.0.23 () {56 vars in 1099 bytes} [Thu May 19 11:57:34 2022] GET /identity/v3/roles/member => generated 84 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 11:57:34.954046 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a25955-0bb6-45e5-aafc-7a37bb7b64de None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:57:34.969518 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547cdb4a-27fa-44c2-b7fb-ad014dd6ec04 None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:57:34.973544 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:57:35.019213 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d3dc89-b19d-434b-b770-6995717a21df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:57:35.019806 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d3dc89-b19d-434b-b770-6995717a21df None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:35.020095 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d3dc89-b19d-434b-b770-6995717a21df None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:35.020372 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d3dc89-b19d-434b-b770-6995717a21df None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:35.021221 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d3dc89-b19d-434b-b770-6995717a21df None admin] RBAC: Adding query filter params (role.id=a49254f8009c463e88e5e0b3909c5904, user.id=07e60193e21e42f4bd442f0292b93267, scope.system=all) {{(pid=80539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 11:57:35.021516 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d3dc89-b19d-434b-b770-6995717a21df None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:57:35.022970 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d3dc89-b19d-434b-b770-6995717a21df None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:57:35.029271 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 149/307] 10.222.0.23 () {56 vars in 1308 bytes} [Thu May 19 11:57:34 2022] GET /identity/v3/role_assignments?user.id=07e60193e21e42f4bd442f0292b93267&scope.system=all&role.id=a49254f8009c463e88e5e0b3909c5904 => generated 226 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 11:57:35.082556 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a25955-0bb6-45e5-aafc-7a37bb7b64de None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:57:35.083117 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1421eb-6470-48f0-83cd-6fd1b11b2406 None None] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:35.083117 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1421eb-6470-48f0-83cd-6fd1b11b2406 None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:35.083117 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1421eb-6470-48f0-83cd-6fd1b11b2406 None None] PATH_INFO: `/` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:35.084861 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 150/308] 10.222.0.23 () {54 vars in 926 bytes} [Thu May 19 11:57:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 11:57:35.090577 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68da88c7-08e0-4a8b-8245-d9dc8c6266e7 None None] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:35.090865 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68da88c7-08e0-4a8b-8245-d9dc8c6266e7 None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:35.091150 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68da88c7-08e0-4a8b-8245-d9dc8c6266e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:35.113081 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.auth.core [None req-68da88c7-08e0-4a8b-8245-d9dc8c6266e7 None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 11:57:35.114566 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-68da88c7-08e0-4a8b-8245-d9dc8c6266e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 11:57:35.129466 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a71eeed-81c9-4fbb-b7e5-8f72b60149bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:57:35.130335 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a71eeed-81c9-4fbb-b7e5-8f72b60149bd None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:35.130722 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a71eeed-81c9-4fbb-b7e5-8f72b60149bd None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:35.131103 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a71eeed-81c9-4fbb-b7e5-8f72b60149bd None admin] PATH_INFO: `/v3/roles` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:35.132195 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a71eeed-81c9-4fbb-b7e5-8f72b60149bd None admin] RBAC: Adding query filter params (name=member) {{(pid=80540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 11:57:35.132535 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a71eeed-81c9-4fbb-b7e5-8f72b60149bd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:57:35.134218 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a71eeed-81c9-4fbb-b7e5-8f72b60149bd None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:57:35.145647 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 159/309] 10.222.0.23 () {56 vars in 1101 bytes} [Thu May 19 11:57:34 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:57:35.151029 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a71eeed-81c9-4fbb-b7e5-8f72b60149bd None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:57:35.222983 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 151/310] 10.222.0.23 () {58 vars in 1023 bytes} [Thu May 19 11:57:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 11:57:35.246625 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68da88c7-08e0-4a8b-8245-d9dc8c6266e7 None None] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:57:35.254315 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:57:35.285838 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a71eeed-81c9-4fbb-b7e5-8f72b60149bd None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:57:35.293111 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:57:35.333643 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb60dfd-27b3-405a-b421-428f10dacfcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:57:35.334901 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb60dfd-27b3-405a-b421-428f10dacfcc None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:35.335634 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb60dfd-27b3-405a-b421-428f10dacfcc None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:35.336154 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb60dfd-27b3-405a-b421-428f10dacfcc None admin] PATH_INFO: `/v3/users/2f54e6b77e7f4ffd8c0a2a2105855fae` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:35.338478 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb60dfd-27b3-405a-b421-428f10dacfcc None admin] RBAC: Authorizing `identity:get_user(user_id=2f54e6b77e7f4ffd8c0a2a2105855fae)` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:57:35.341503 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb60dfd-27b3-405a-b421-428f10dacfcc None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:57:35.343762 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 160/311] 10.222.0.23 () {56 vars in 1177 bytes} [Thu May 19 11:57:35 2022] GET /identity/v3/users/2f54e6b77e7f4ffd8c0a2a2105855fae => generated 293 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 11:57:35.349280 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb60dfd-27b3-405a-b421-428f10dacfcc None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:57:35.385464 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68da88c7-08e0-4a8b-8245-d9dc8c6266e7 None None] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:57:35.430051 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a9034f-9ad3-424f-a12e-4b5e4b853add None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:57:35.430869 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a9034f-9ad3-424f-a12e-4b5e4b853add None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:35.431154 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a9034f-9ad3-424f-a12e-4b5e4b853add None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:35.431433 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a9034f-9ad3-424f-a12e-4b5e4b853add None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:35.445060 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a9034f-9ad3-424f-a12e-4b5e4b853add None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 11:57:35.453832 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application [None req-f8a9034f-9ad3-424f-a12e-4b5e4b853add None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 19 11:57:35.453832 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 11:57:35.453832 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 11:57:35.453832 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 11:57:35.453832 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 11:57:35.453832 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 11:57:35.453832 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 11:57:35.453832 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 11:57:35.453832 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 11:57:35.453832 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 11:57:35.453832 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 11:57:35.453832 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 11:57:35.453832 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 11:57:35.453832 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 11:57:35.453832 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 11:57:35.453832 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 11:57:35.454735 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 11:57:35.454735 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 11:57:35.454735 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 11:57:35.454735 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 11:57:35.454735 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 11:57:35.454735 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 11:57:35.454735 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 11:57:35.454735 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 11:57:35.454735 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 11:57:35.454735 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.get_or_create( May 19 11:57:35.454735 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 11:57:35.454735 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application with Lock( May 19 11:57:35.454735 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 11:57:35.454735 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._enter() May 19 11:57:35.454735 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 11:57:35.454735 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 11:57:35.455515 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 11:57:35.455515 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.creator() May 19 11:57:35.455515 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 11:57:35.455515 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application created_value = creator( May 19 11:57:35.455515 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 11:57:35.455515 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 11:57:35.455515 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 11:57:35.455515 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 11:57:35.455515 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 11:57:35.455515 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 11:57:35.455515 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 19 11:57:35.455515 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application May 19 11:57:35.455515 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 152/312] 10.222.0.23 () {56 vars in 1099 bytes} [Thu May 19 11:57:35 2022] GET /identity/v3/roles/reader => generated 84 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 11:57:35.461320 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a9034f-9ad3-424f-a12e-4b5e4b853add None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:57:35.483061 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb60dfd-27b3-405a-b421-428f10dacfcc None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:57:35.531553 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01120bbe-5cf3-4b7e-a626-27a17bcda2be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:57:35.533046 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01120bbe-5cf3-4b7e-a626-27a17bcda2be None admin] REQUEST_METHOD: `PUT` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:35.534134 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01120bbe-5cf3-4b7e-a626-27a17bcda2be None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:35.534618 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01120bbe-5cf3-4b7e-a626-27a17bcda2be None admin] PATH_INFO: `/v3/system/users/2f54e6b77e7f4ffd8c0a2a2105855fae/roles/259eb400cdb4439db72d14456ae9cd08` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:35.537145 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01120bbe-5cf3-4b7e-a626-27a17bcda2be None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=2f54e6b77e7f4ffd8c0a2a2105855fae, role_id=259eb400cdb4439db72d14456ae9cd08)` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:57:35.539979 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01120bbe-5cf3-4b7e-a626-27a17bcda2be None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:57:35.561460 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:57:35.575674 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a9034f-9ad3-424f-a12e-4b5e4b853add None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:57:35.611859 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f90574-f8ea-46a2-934c-d8d455302d7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:57:35.612531 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f90574-f8ea-46a2-934c-d8d455302d7d None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:35.612815 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f90574-f8ea-46a2-934c-d8d455302d7d None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:35.613093 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f90574-f8ea-46a2-934c-d8d455302d7d None admin] PATH_INFO: `/v3/roles` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:35.613841 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f90574-f8ea-46a2-934c-d8d455302d7d None admin] RBAC: Adding query filter params (name=reader) {{(pid=80539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 11:57:35.614149 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f90574-f8ea-46a2-934c-d8d455302d7d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:57:35.615707 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f90574-f8ea-46a2-934c-d8d455302d7d None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:57:35.628154 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 153/313] 10.222.0.23 () {56 vars in 1101 bytes} [Thu May 19 11:57:35 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 11:57:35.630908 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f90574-f8ea-46a2-934c-d8d455302d7d None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:57:35.636930 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 161/314] 10.222.0.23 () {58 vars in 1334 bytes} [Thu May 19 11:57:35 2022] PUT /identity/v3/system/users/2f54e6b77e7f4ffd8c0a2a2105855fae/roles/259eb400cdb4439db72d14456ae9cd08 => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 11:57:35.758035 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f90574-f8ea-46a2-934c-d8d455302d7d None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:57:35.798554 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c73129c-6cef-4836-8cac-7c9cf73fa355 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:57:35.799357 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c73129c-6cef-4836-8cac-7c9cf73fa355 None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:35.799637 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c73129c-6cef-4836-8cac-7c9cf73fa355 None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:35.799947 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c73129c-6cef-4836-8cac-7c9cf73fa355 None admin] PATH_INFO: `/v3/users/0b74c769188e46608374f314768d312a` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:35.801256 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c73129c-6cef-4836-8cac-7c9cf73fa355 None admin] RBAC: Authorizing `identity:get_user(user_id=0b74c769188e46608374f314768d312a)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:57:35.802741 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c73129c-6cef-4836-8cac-7c9cf73fa355 None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:57:35.809235 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 154/315] 10.222.0.23 () {56 vars in 1177 bytes} [Thu May 19 11:57:35 2022] GET /identity/v3/users/0b74c769188e46608374f314768d312a => generated 297 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 11:57:35.810764 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01120bbe-5cf3-4b7e-a626-27a17bcda2be None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:57:35.824899 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:57:35.923066 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01120bbe-5cf3-4b7e-a626-27a17bcda2be None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:57:35.960491 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28795bf-2aa3-4821-b1ba-cb7c27ed6d83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:57:35.961394 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28795bf-2aa3-4821-b1ba-cb7c27ed6d83 None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:35.961854 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28795bf-2aa3-4821-b1ba-cb7c27ed6d83 None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:35.962319 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28795bf-2aa3-4821-b1ba-cb7c27ed6d83 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:36.039233 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28795bf-2aa3-4821-b1ba-cb7c27ed6d83 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:57:36.040801 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28795bf-2aa3-4821-b1ba-cb7c27ed6d83 None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:57:36.054465 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:57:36.149439 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 162/316] 10.222.0.23 () {58 vars in 1282 bytes} [Thu May 19 11:57:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 11:57:36.155996 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c73129c-6cef-4836-8cac-7c9cf73fa355 None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:57:36.179922 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:57:36.268007 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c73129c-6cef-4836-8cac-7c9cf73fa355 None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:57:36.304344 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0521cc-4287-44cc-b59d-ede52265960c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:57:36.305420 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0521cc-4287-44cc-b59d-ede52265960c None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:36.305634 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0521cc-4287-44cc-b59d-ede52265960c None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:36.305908 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0521cc-4287-44cc-b59d-ede52265960c None admin] PATH_INFO: `/v3/projects/0c5d68529f48460f9f6e03e3f8fb205b` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:36.307105 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0521cc-4287-44cc-b59d-ede52265960c None admin] RBAC: Authorizing `identity:get_project(project_id=0c5d68529f48460f9f6e03e3f8fb205b)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:57:36.308586 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0521cc-4287-44cc-b59d-ede52265960c None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:57:36.310459 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 155/317] 10.222.0.23 () {56 vars in 1186 bytes} [Thu May 19 11:57:36 2022] GET /identity/v3/projects/0c5d68529f48460f9f6e03e3f8fb205b => generated 302 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:57:36.321007 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28795bf-2aa3-4821-b1ba-cb7c27ed6d83 None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:57:36.425238 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28795bf-2aa3-4821-b1ba-cb7c27ed6d83 None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:57:36.435834 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:57:36.462174 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e756936a-f219-4732-bc55-7c2d94bf85c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:57:36.462833 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e756936a-f219-4732-bc55-7c2d94bf85c9 None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:36.463119 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e756936a-f219-4732-bc55-7c2d94bf85c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:36.463397 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e756936a-f219-4732-bc55-7c2d94bf85c9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:36.464270 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e756936a-f219-4732-bc55-7c2d94bf85c9 None admin] RBAC: Adding query filter params (role.id=a49254f8009c463e88e5e0b3909c5904, scope.project.id=0c5d68529f48460f9f6e03e3f8fb205b, user.id=0b74c769188e46608374f314768d312a) {{(pid=80540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 11:57:36.464585 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e756936a-f219-4732-bc55-7c2d94bf85c9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:57:36.466061 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e756936a-f219-4732-bc55-7c2d94bf85c9 None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:57:36.480036 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 163/318] 10.222.0.23 () {56 vars in 1374 bytes} [Thu May 19 11:57:36 2022] GET /identity/v3/role_assignments?user.id=0b74c769188e46608374f314768d312a&scope.project.id=0c5d68529f48460f9f6e03e3f8fb205b&role.id=a49254f8009c463e88e5e0b3909c5904 => generated 608 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:57:37.178609 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f5a364-67ea-412f-99ed-3b6f2f75af66 None None] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:37.179188 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f5a364-67ea-412f-99ed-3b6f2f75af66 None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:37.179188 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f5a364-67ea-412f-99ed-3b6f2f75af66 None None] PATH_INFO: `/` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:37.180706 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 156/319] 10.222.0.23 () {54 vars in 926 bytes} [Thu May 19 11:57:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 11:57:37.186059 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e24e30e-d9ba-4e8d-8f1b-33224f8ff32e None None] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:37.186545 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e24e30e-d9ba-4e8d-8f1b-33224f8ff32e None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:37.186834 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e24e30e-d9ba-4e8d-8f1b-33224f8ff32e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:37.206027 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.auth.core [None req-9e24e30e-d9ba-4e8d-8f1b-33224f8ff32e None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 11:57:37.207161 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-9e24e30e-d9ba-4e8d-8f1b-33224f8ff32e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 11:57:37.283825 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 164/320] 10.222.0.23 () {58 vars in 1023 bytes} [Thu May 19 11:57:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 11:57:37.305297 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f5a364-67ea-412f-99ed-3b6f2f75af66 None None] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:57:37.398626 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:57:37.406331 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5561cc66-dcdd-40a1-aa3d-701651b3a725 None None] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:37.406626 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5561cc66-dcdd-40a1-aa3d-701651b3a725 None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:37.406912 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5561cc66-dcdd-40a1-aa3d-701651b3a725 None None] PATH_INFO: `/` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:37.408466 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 165/321] 10.222.0.23 () {54 vars in 926 bytes} [Thu May 19 11:57:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 11:57:37.413243 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d71f5e4-1a10-47a0-b178-15eb4974b1ae None None] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:37.421226 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d71f5e4-1a10-47a0-b178-15eb4974b1ae None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:37.421226 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d71f5e4-1a10-47a0-b178-15eb4974b1ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:37.421516 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f5a364-67ea-412f-99ed-3b6f2f75af66 None None] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:57:37.433392 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.auth.core [None req-8d71f5e4-1a10-47a0-b178-15eb4974b1ae None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 11:57:37.435255 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-8d71f5e4-1a10-47a0-b178-15eb4974b1ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 11:57:37.451577 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e672e03a-c34a-4284-8bcf-59f01a718973 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:57:37.452892 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e672e03a-c34a-4284-8bcf-59f01a718973 None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:37.453330 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e672e03a-c34a-4284-8bcf-59f01a718973 None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:37.453475 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e672e03a-c34a-4284-8bcf-59f01a718973 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:37.456615 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e672e03a-c34a-4284-8bcf-59f01a718973 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:57:37.458005 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e672e03a-c34a-4284-8bcf-59f01a718973 None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:57:37.459808 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 157/322] 10.222.0.23 () {56 vars in 1108 bytes} [Thu May 19 11:57:37 2022] GET /identity/v3/domains/default => generated 199 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:57:37.464993 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e672e03a-c34a-4284-8bcf-59f01a718973 None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:57:37.520149 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 166/323] 10.222.0.23 () {58 vars in 1023 bytes} [Thu May 19 11:57:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 11:57:37.542761 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d71f5e4-1a10-47a0-b178-15eb4974b1ae None None] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:57:37.574047 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e672e03a-c34a-4284-8bcf-59f01a718973 None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:57:37.575321 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:57:37.613908 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c05bce2-be7e-404e-b73c-914a3bbbf85c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:57:37.614670 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c05bce2-be7e-404e-b73c-914a3bbbf85c None admin] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:37.614953 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c05bce2-be7e-404e-b73c-914a3bbbf85c None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:37.615233 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c05bce2-be7e-404e-b73c-914a3bbbf85c None admin] PATH_INFO: `/v3/groups` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:37.615963 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c05bce2-be7e-404e-b73c-914a3bbbf85c None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:57:37.617408 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c05bce2-be7e-404e-b73c-914a3bbbf85c None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:57:37.649150 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d71f5e4-1a10-47a0-b178-15eb4974b1ae None None] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:57:37.684251 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b479cb-0c14-46db-8287-c98700ed645c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:57:37.685053 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b479cb-0c14-46db-8287-c98700ed645c None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:37.685339 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b479cb-0c14-46db-8287-c98700ed645c None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:37.685634 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b479cb-0c14-46db-8287-c98700ed645c None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:37.697381 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b479cb-0c14-46db-8287-c98700ed645c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 11:57:37.697594 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 158/324] 10.222.0.23 () {60 vars in 1134 bytes} [Thu May 19 11:57:37 2022] POST /identity/v3/groups => generated 226 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 11:57:37.705936 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application [None req-06b479cb-0c14-46db-8287-c98700ed645c None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 19 11:57:37.705936 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 11:57:37.705936 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 11:57:37.705936 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 11:57:37.705936 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 11:57:37.705936 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 11:57:37.705936 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 11:57:37.705936 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 11:57:37.705936 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 11:57:37.705936 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 11:57:37.705936 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 11:57:37.705936 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 11:57:37.705936 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 11:57:37.705936 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 11:57:37.705936 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 11:57:37.705936 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 11:57:37.706968 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 11:57:37.706968 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 11:57:37.706968 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 11:57:37.706968 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 11:57:37.706968 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 11:57:37.706968 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 11:57:37.706968 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 11:57:37.706968 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 11:57:37.706968 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 11:57:37.706968 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.get_or_create( May 19 11:57:37.706968 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 11:57:37.706968 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application with Lock( May 19 11:57:37.706968 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 11:57:37.706968 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._enter() May 19 11:57:37.706968 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 11:57:37.706968 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 11:57:37.707759 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 11:57:37.707759 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.creator() May 19 11:57:37.707759 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 11:57:37.707759 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application created_value = creator( May 19 11:57:37.707759 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 11:57:37.707759 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 11:57:37.707759 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 11:57:37.707759 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 11:57:37.707759 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 11:57:37.707759 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 11:57:37.707759 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 19 11:57:37.707759 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application May 19 11:57:37.712217 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 167/325] 10.222.0.23 () {56 vars in 1099 bytes} [Thu May 19 11:57:37 2022] GET /identity/v3/roles/reader => generated 84 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 11:57:37.713646 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c05bce2-be7e-404e-b73c-914a3bbbf85c None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:57:37.736614 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:57:37.833406 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c05bce2-be7e-404e-b73c-914a3bbbf85c None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:57:37.871721 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f6c4d5-8001-4627-a2b0-f35d0b87e419 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:57:37.872449 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f6c4d5-8001-4627-a2b0-f35d0b87e419 None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:37.872732 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f6c4d5-8001-4627-a2b0-f35d0b87e419 None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:37.873008 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f6c4d5-8001-4627-a2b0-f35d0b87e419 None admin] PATH_INFO: `/v3/roles` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:37.873771 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f6c4d5-8001-4627-a2b0-f35d0b87e419 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 11:57:37.874616 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f6c4d5-8001-4627-a2b0-f35d0b87e419 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:57:37.876056 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f6c4d5-8001-4627-a2b0-f35d0b87e419 None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:57:37.890295 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 159/326] 10.222.0.23 () {56 vars in 1101 bytes} [Thu May 19 11:57:37 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:57:37.891846 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b479cb-0c14-46db-8287-c98700ed645c None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:57:37.993597 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1927e732-fbe4-4e49-9318-597e8e1ef480 None None] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:37.993905 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1927e732-fbe4-4e49-9318-597e8e1ef480 None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:37.994096 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1927e732-fbe4-4e49-9318-597e8e1ef480 None None] PATH_INFO: `/` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:37.995425 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:57:37.997214 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 160/327] 10.222.0.23 () {54 vars in 926 bytes} [Thu May 19 11:57:37 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 11:57:38.005597 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b9edae-dd72-461d-936b-cbd9e397eb8c None None] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:38.013915 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b9edae-dd72-461d-936b-cbd9e397eb8c None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:38.013915 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b9edae-dd72-461d-936b-cbd9e397eb8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:38.014098 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b479cb-0c14-46db-8287-c98700ed645c None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:57:38.028935 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.auth.core [None req-02b9edae-dd72-461d-936b-cbd9e397eb8c None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 11:57:38.033199 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-02b9edae-dd72-461d-936b-cbd9e397eb8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 11:57:38.056156 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc37b558-00d7-41b0-892d-6865fd001726 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:57:38.056156 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc37b558-00d7-41b0-892d-6865fd001726 None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:38.056156 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc37b558-00d7-41b0-892d-6865fd001726 None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:38.056156 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc37b558-00d7-41b0-892d-6865fd001726 None admin] PATH_INFO: `/v3/users/07e60193e21e42f4bd442f0292b93267` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:38.056996 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc37b558-00d7-41b0-892d-6865fd001726 None admin] RBAC: Authorizing `identity:get_user(user_id=07e60193e21e42f4bd442f0292b93267)` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:57:38.058466 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc37b558-00d7-41b0-892d-6865fd001726 None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:57:38.060980 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 168/328] 10.222.0.23 () {56 vars in 1177 bytes} [Thu May 19 11:57:37 2022] GET /identity/v3/users/07e60193e21e42f4bd442f0292b93267 => generated 293 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:57:38.066460 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc37b558-00d7-41b0-892d-6865fd001726 None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:57:38.114267 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 161/329] 10.222.0.23 () {58 vars in 1023 bytes} [Thu May 19 11:57:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 11:57:38.139150 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b9edae-dd72-461d-936b-cbd9e397eb8c None None] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:57:38.150370 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:57:38.185767 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc37b558-00d7-41b0-892d-6865fd001726 None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:57:38.227857 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14baeaa-fdb1-493f-9683-2bb5e9953851 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:57:38.229467 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14baeaa-fdb1-493f-9683-2bb5e9953851 None admin] REQUEST_METHOD: `PUT` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:38.229942 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14baeaa-fdb1-493f-9683-2bb5e9953851 None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:38.230516 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14baeaa-fdb1-493f-9683-2bb5e9953851 None admin] PATH_INFO: `/v3/system/users/07e60193e21e42f4bd442f0292b93267/roles/a49254f8009c463e88e5e0b3909c5904` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:38.232207 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14baeaa-fdb1-493f-9683-2bb5e9953851 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=07e60193e21e42f4bd442f0292b93267, role_id=a49254f8009c463e88e5e0b3909c5904)` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:57:38.234216 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14baeaa-fdb1-493f-9683-2bb5e9953851 None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:57:38.265430 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b9edae-dd72-461d-936b-cbd9e397eb8c None None] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:57:38.297576 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 169/330] 10.222.0.23 () {58 vars in 1334 bytes} [Thu May 19 11:57:38 2022] PUT /identity/v3/system/users/07e60193e21e42f4bd442f0292b93267/roles/a49254f8009c463e88e5e0b3909c5904 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 11:57:38.318635 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94c160c-9c2b-49a2-8d56-a0044d9fda7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:57:38.319441 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94c160c-9c2b-49a2-8d56-a0044d9fda7d None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:38.319986 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94c160c-9c2b-49a2-8d56-a0044d9fda7d None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:38.320301 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94c160c-9c2b-49a2-8d56-a0044d9fda7d None admin] PATH_INFO: `/v3/roles/member` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:38.338352 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94c160c-9c2b-49a2-8d56-a0044d9fda7d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 11:57:38.348086 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application [None req-b94c160c-9c2b-49a2-8d56-a0044d9fda7d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 11:57:38.348086 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 11:57:38.348086 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 11:57:38.348086 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 11:57:38.348086 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 11:57:38.348086 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 11:57:38.348086 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 11:57:38.348086 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 11:57:38.348086 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 11:57:38.348086 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 11:57:38.348086 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 11:57:38.348086 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 11:57:38.348086 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 11:57:38.348086 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 11:57:38.348086 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 11:57:38.348086 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 11:57:38.349056 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 11:57:38.349056 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 11:57:38.349056 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 11:57:38.349056 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 11:57:38.349056 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 11:57:38.349056 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 11:57:38.349056 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 11:57:38.349056 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 11:57:38.349056 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 11:57:38.349056 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.get_or_create( May 19 11:57:38.349056 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 11:57:38.349056 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application with Lock( May 19 11:57:38.349056 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 11:57:38.349056 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._enter() May 19 11:57:38.349056 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 11:57:38.349056 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 11:57:38.349835 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 11:57:38.349835 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.creator() May 19 11:57:38.349835 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 11:57:38.349835 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application created_value = creator( May 19 11:57:38.349835 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 11:57:38.349835 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 11:57:38.349835 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 11:57:38.349835 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 11:57:38.349835 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 11:57:38.349835 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 11:57:38.349835 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 11:57:38.349835 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application May 19 11:57:38.352532 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 162/331] 10.222.0.23 () {56 vars in 1099 bytes} [Thu May 19 11:57:38 2022] GET /identity/v3/roles/member => generated 84 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 11:57:38.358597 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14baeaa-fdb1-493f-9683-2bb5e9953851 None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:57:38.401382 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:57:38.499758 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14baeaa-fdb1-493f-9683-2bb5e9953851 None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:57:38.540883 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85385ad2-8793-411d-a1ec-b938ba4967e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:57:38.541895 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85385ad2-8793-411d-a1ec-b938ba4967e8 None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:38.542185 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85385ad2-8793-411d-a1ec-b938ba4967e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:38.542459 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85385ad2-8793-411d-a1ec-b938ba4967e8 None admin] PATH_INFO: `/v3/roles` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:38.543209 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85385ad2-8793-411d-a1ec-b938ba4967e8 None admin] RBAC: Adding query filter params (name=member) {{(pid=80540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 11:57:38.543517 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85385ad2-8793-411d-a1ec-b938ba4967e8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:57:38.544958 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85385ad2-8793-411d-a1ec-b938ba4967e8 None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:57:38.556401 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 170/332] 10.222.0.23 () {56 vars in 1101 bytes} [Thu May 19 11:57:38 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:57:38.561901 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94c160c-9c2b-49a2-8d56-a0044d9fda7d None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:57:38.650301 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:57:38.688487 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94c160c-9c2b-49a2-8d56-a0044d9fda7d None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:57:38.726183 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7fe2a6-746a-4c6e-8998-0fd79f5acca9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:57:38.727061 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7fe2a6-746a-4c6e-8998-0fd79f5acca9 None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:38.727343 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7fe2a6-746a-4c6e-8998-0fd79f5acca9 None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:38.727616 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7fe2a6-746a-4c6e-8998-0fd79f5acca9 None admin] PATH_INFO: `/v3/users/2f54e6b77e7f4ffd8c0a2a2105855fae` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:38.728867 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7fe2a6-746a-4c6e-8998-0fd79f5acca9 None admin] RBAC: Authorizing `identity:get_user(user_id=2f54e6b77e7f4ffd8c0a2a2105855fae)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:57:38.730323 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7fe2a6-746a-4c6e-8998-0fd79f5acca9 None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:57:38.732273 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 163/333] 10.222.0.23 () {56 vars in 1177 bytes} [Thu May 19 11:57:38 2022] GET /identity/v3/users/2f54e6b77e7f4ffd8c0a2a2105855fae => generated 293 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:57:38.738568 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85385ad2-8793-411d-a1ec-b938ba4967e8 None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:57:38.858236 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:57:38.875599 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85385ad2-8793-411d-a1ec-b938ba4967e8 None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:57:38.915129 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb5de85-bba0-4c3f-8f0c-df8a33061a91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:57:38.915993 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb5de85-bba0-4c3f-8f0c-df8a33061a91 None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:38.916280 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb5de85-bba0-4c3f-8f0c-df8a33061a91 None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:38.916587 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb5de85-bba0-4c3f-8f0c-df8a33061a91 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:38.917393 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb5de85-bba0-4c3f-8f0c-df8a33061a91 None admin] RBAC: Adding query filter params (role.id=259eb400cdb4439db72d14456ae9cd08, user.id=2f54e6b77e7f4ffd8c0a2a2105855fae, scope.system=all) {{(pid=80540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 11:57:38.917782 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb5de85-bba0-4c3f-8f0c-df8a33061a91 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:57:38.919252 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb5de85-bba0-4c3f-8f0c-df8a33061a91 None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:57:38.936524 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 171/334] 10.222.0.23 () {56 vars in 1308 bytes} [Thu May 19 11:57:38 2022] GET /identity/v3/role_assignments?user.id=2f54e6b77e7f4ffd8c0a2a2105855fae&scope.system=all&role.id=259eb400cdb4439db72d14456ae9cd08 => generated 510 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:57:40.985843 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70fd383-0522-4b83-9551-fae6d24dbffe None None] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:40.986834 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70fd383-0522-4b83-9551-fae6d24dbffe None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:40.987316 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70fd383-0522-4b83-9551-fae6d24dbffe None None] PATH_INFO: `/` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:40.989087 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 164/335] 10.222.0.23 () {54 vars in 926 bytes} [Thu May 19 11:57:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 11:57:40.996235 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1a57db-484d-4b34-b4ce-18d321ad287d None None] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:40.997223 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1a57db-484d-4b34-b4ce-18d321ad287d None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:40.997705 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1a57db-484d-4b34-b4ce-18d321ad287d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:41.017265 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.auth.core [None req-7b1a57db-484d-4b34-b4ce-18d321ad287d None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 11:57:41.018791 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-7b1a57db-484d-4b34-b4ce-18d321ad287d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 11:57:41.095432 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 172/336] 10.222.0.23 () {58 vars in 1023 bytes} [Thu May 19 11:57:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 11:57:41.119174 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70fd383-0522-4b83-9551-fae6d24dbffe None None] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:57:41.188028 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f7a4ef-df94-4849-97e3-55cb107067aa None None] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:41.188657 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f7a4ef-df94-4849-97e3-55cb107067aa None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:41.189128 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f7a4ef-df94-4849-97e3-55cb107067aa None None] PATH_INFO: `/` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:41.190998 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 173/337] 10.222.0.23 () {54 vars in 926 bytes} [Thu May 19 11:57:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 11:57:41.196211 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2890a81e-7acd-44a2-ad87-7e2d11568b2a None None] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:41.196768 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2890a81e-7acd-44a2-ad87-7e2d11568b2a None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:41.197266 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2890a81e-7acd-44a2-ad87-7e2d11568b2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:41.225556 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.auth.core [None req-2890a81e-7acd-44a2-ad87-7e2d11568b2a None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 11:57:41.227729 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-2890a81e-7acd-44a2-ad87-7e2d11568b2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 11:57:41.312216 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 174/338] 10.222.0.23 () {58 vars in 1023 bytes} [Thu May 19 11:57:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 11:57:41.315223 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70fd383-0522-4b83-9551-fae6d24dbffe None None] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:57:41.371386 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85da9f24-0e0f-4cf9-a164-1353c126ed5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:57:41.372233 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85da9f24-0e0f-4cf9-a164-1353c126ed5a None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:41.372524 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85da9f24-0e0f-4cf9-a164-1353c126ed5a None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:41.372806 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85da9f24-0e0f-4cf9-a164-1353c126ed5a None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:41.377072 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:57:41.383969 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2890a81e-7acd-44a2-ad87-7e2d11568b2a None None] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:57:41.387437 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:57:41.391460 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85da9f24-0e0f-4cf9-a164-1353c126ed5a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 11:57:41.399296 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application [None req-85da9f24-0e0f-4cf9-a164-1353c126ed5a None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 19 11:57:41.399296 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 11:57:41.399296 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 11:57:41.399296 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 11:57:41.399296 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 11:57:41.399296 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 11:57:41.399296 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 11:57:41.399296 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 11:57:41.399296 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 11:57:41.399296 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 11:57:41.399296 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 11:57:41.399296 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 11:57:41.399296 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 11:57:41.399296 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 11:57:41.399296 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 11:57:41.399296 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 11:57:41.405498 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 11:57:41.405498 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 11:57:41.405498 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 11:57:41.405498 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 11:57:41.405498 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 11:57:41.405498 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 11:57:41.405498 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 11:57:41.405498 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 11:57:41.405498 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 11:57:41.405498 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.get_or_create( May 19 11:57:41.405498 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 11:57:41.405498 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application with Lock( May 19 11:57:41.405498 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 11:57:41.405498 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._enter() May 19 11:57:41.405498 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 11:57:41.405498 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 11:57:41.406408 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 11:57:41.406408 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.creator() May 19 11:57:41.406408 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 11:57:41.406408 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application created_value = creator( May 19 11:57:41.406408 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 11:57:41.406408 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 11:57:41.406408 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 11:57:41.406408 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 11:57:41.406408 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 11:57:41.406408 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 11:57:41.406408 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 19 11:57:41.406408 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application May 19 11:57:41.406408 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 165/339] 10.222.0.23 () {56 vars in 1099 bytes} [Thu May 19 11:57:41 2022] GET /identity/v3/roles/reader => generated 84 bytes in 286 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 11:57:41.424086 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85da9f24-0e0f-4cf9-a164-1353c126ed5a None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:57:41.515662 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2890a81e-7acd-44a2-ad87-7e2d11568b2a None None] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:57:41.552655 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa66dfa-bc72-4a0c-990c-1df9a13c3235 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:57:41.553703 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa66dfa-bc72-4a0c-990c-1df9a13c3235 None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:41.554149 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa66dfa-bc72-4a0c-990c-1df9a13c3235 None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:41.554572 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa66dfa-bc72-4a0c-990c-1df9a13c3235 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:41.558043 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85da9f24-0e0f-4cf9-a164-1353c126ed5a None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:57:41.566255 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa66dfa-bc72-4a0c-990c-1df9a13c3235 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 11:57:41.574470 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application [None req-0aa66dfa-bc72-4a0c-990c-1df9a13c3235 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 11:57:41.574470 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 11:57:41.574470 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 11:57:41.574470 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 11:57:41.574470 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 11:57:41.574470 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 11:57:41.574470 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 11:57:41.574470 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 11:57:41.574470 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 11:57:41.574470 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 11:57:41.574470 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 11:57:41.574470 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 11:57:41.574470 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 11:57:41.574470 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 11:57:41.574470 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 11:57:41.574470 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 11:57:41.575533 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 11:57:41.575533 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 11:57:41.575533 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 11:57:41.575533 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 11:57:41.575533 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 11:57:41.575533 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 11:57:41.575533 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 11:57:41.575533 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 11:57:41.575533 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 11:57:41.575533 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.get_or_create( May 19 11:57:41.575533 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 11:57:41.575533 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application with Lock( May 19 11:57:41.575533 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 11:57:41.575533 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._enter() May 19 11:57:41.575533 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 11:57:41.575533 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 11:57:41.576546 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 11:57:41.576546 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.creator() May 19 11:57:41.576546 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 11:57:41.576546 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application created_value = creator( May 19 11:57:41.576546 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 11:57:41.576546 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 11:57:41.576546 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 11:57:41.576546 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 11:57:41.576546 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 11:57:41.576546 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 11:57:41.576546 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 11:57:41.576546 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application May 19 11:57:41.578982 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 175/340] 10.222.0.23 () {56 vars in 1099 bytes} [Thu May 19 11:57:41 2022] GET /identity/v3/roles/member => generated 84 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 11:57:41.580510 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d549398-21aa-4211-9ba4-030477b6b780 None None] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:41.580972 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d549398-21aa-4211-9ba4-030477b6b780 None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:41.581410 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d549398-21aa-4211-9ba4-030477b6b780 None None] PATH_INFO: `/` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:41.582922 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 176/341] 10.222.0.23 () {54 vars in 926 bytes} [Thu May 19 11:57:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 11:57:41.589251 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e252430-de3b-49b5-9769-7c5b9713dbe5 None None] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:41.589806 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e252430-de3b-49b5-9769-7c5b9713dbe5 None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:41.590297 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e252430-de3b-49b5-9769-7c5b9713dbe5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:41.605395 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42aa8f4c-a495-40da-b235-0232b5459701 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:57:41.606078 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42aa8f4c-a495-40da-b235-0232b5459701 None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:41.606360 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42aa8f4c-a495-40da-b235-0232b5459701 None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:41.606632 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42aa8f4c-a495-40da-b235-0232b5459701 None admin] PATH_INFO: `/v3/roles` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:41.607387 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42aa8f4c-a495-40da-b235-0232b5459701 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 11:57:41.607689 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42aa8f4c-a495-40da-b235-0232b5459701 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:57:41.609891 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42aa8f4c-a495-40da-b235-0232b5459701 None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:57:41.615373 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.auth.core [None req-6e252430-de3b-49b5-9769-7c5b9713dbe5 None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 11:57:41.616979 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-6e252430-de3b-49b5-9769-7c5b9713dbe5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 11:57:41.629999 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 166/342] 10.222.0.23 () {56 vars in 1101 bytes} [Thu May 19 11:57:41 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 11:57:41.630876 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42aa8f4c-a495-40da-b235-0232b5459701 None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:57:41.688702 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:57:41.697746 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 177/343] 10.222.0.23 () {58 vars in 1023 bytes} [Thu May 19 11:57:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 11:57:41.698837 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e252430-de3b-49b5-9769-7c5b9713dbe5 None None] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:57:41.727449 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:57:41.778368 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42aa8f4c-a495-40da-b235-0232b5459701 None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:57:41.817711 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a900095d-e7a5-4bf7-9577-ddd54554a7ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:57:41.818709 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a900095d-e7a5-4bf7-9577-ddd54554a7ec None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:41.819240 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a900095d-e7a5-4bf7-9577-ddd54554a7ec None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:41.819961 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a900095d-e7a5-4bf7-9577-ddd54554a7ec None admin] PATH_INFO: `/v3/roles` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:41.820940 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a900095d-e7a5-4bf7-9577-ddd54554a7ec None admin] RBAC: Adding query filter params (name=member) {{(pid=80539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 11:57:41.821434 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a900095d-e7a5-4bf7-9577-ddd54554a7ec None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:57:41.823028 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a900095d-e7a5-4bf7-9577-ddd54554a7ec None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:57:41.832656 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 167/344] 10.222.0.23 () {56 vars in 1101 bytes} [Thu May 19 11:57:41 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:57:41.834421 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72845b78-6b01-4176-a73d-332301f672bf None None] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:41.834936 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72845b78-6b01-4176-a73d-332301f672bf None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:41.835408 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72845b78-6b01-4176-a73d-332301f672bf None None] PATH_INFO: `/` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:41.836951 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 168/345] 10.222.0.23 () {54 vars in 926 bytes} [Thu May 19 11:57:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 11:57:41.837893 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72845b78-6b01-4176-a73d-332301f672bf None None] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:57:41.854945 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e252430-de3b-49b5-9769-7c5b9713dbe5 None None] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:57:41.900535 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8257dd32-7353-45fe-8f9c-c161ca7f0bc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:57:41.901850 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8257dd32-7353-45fe-8f9c-c161ca7f0bc1 None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:41.902217 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8257dd32-7353-45fe-8f9c-c161ca7f0bc1 None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:41.902559 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8257dd32-7353-45fe-8f9c-c161ca7f0bc1 None admin] PATH_INFO: `/v3/users/07e60193e21e42f4bd442f0292b93267` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:41.904536 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8257dd32-7353-45fe-8f9c-c161ca7f0bc1 None admin] RBAC: Authorizing `identity:get_user(user_id=07e60193e21e42f4bd442f0292b93267)` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:57:41.907534 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8257dd32-7353-45fe-8f9c-c161ca7f0bc1 None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:57:41.909753 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 178/346] 10.222.0.23 () {56 vars in 1177 bytes} [Thu May 19 11:57:41 2022] GET /identity/v3/users/07e60193e21e42f4bd442f0292b93267 => generated 293 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:57:41.913481 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdbc692-34c3-43e7-b338-61f14b77f3aa None None] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:41.916168 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdbc692-34c3-43e7-b338-61f14b77f3aa None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:41.917352 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdbc692-34c3-43e7-b338-61f14b77f3aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:41.952734 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.auth.core [None req-afdbc692-34c3-43e7-b338-61f14b77f3aa None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 11:57:41.953987 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-afdbc692-34c3-43e7-b338-61f14b77f3aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 11:57:41.997813 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72845b78-6b01-4176-a73d-332301f672bf None None] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:57:42.032328 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:57:42.034500 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:57:42.048429 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946cdbce-0fe1-4083-9bb2-b00014352ea0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:57:42.049657 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946cdbce-0fe1-4083-9bb2-b00014352ea0 None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:42.050094 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946cdbce-0fe1-4083-9bb2-b00014352ea0 None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:42.050415 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 179/347] 10.222.0.23 () {58 vars in 1023 bytes} [Thu May 19 11:57:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 11:57:42.050671 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946cdbce-0fe1-4083-9bb2-b00014352ea0 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:42.051461 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afdbc692-34c3-43e7-b338-61f14b77f3aa None None] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:57:42.067156 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946cdbce-0fe1-4083-9bb2-b00014352ea0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 11:57:42.077931 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application [None req-946cdbce-0fe1-4083-9bb2-b00014352ea0 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 11:57:42.077931 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 11:57:42.077931 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 11:57:42.077931 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 11:57:42.077931 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 11:57:42.077931 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 11:57:42.077931 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 11:57:42.077931 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 11:57:42.077931 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 11:57:42.077931 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 11:57:42.077931 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 11:57:42.077931 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 11:57:42.077931 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 11:57:42.077931 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 11:57:42.077931 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 11:57:42.077931 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 11:57:42.079026 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 11:57:42.079026 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 11:57:42.079026 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 11:57:42.079026 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 11:57:42.079026 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 11:57:42.079026 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 11:57:42.079026 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 11:57:42.079026 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 11:57:42.079026 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 11:57:42.079026 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.get_or_create( May 19 11:57:42.079026 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 11:57:42.079026 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application with Lock( May 19 11:57:42.079026 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 11:57:42.079026 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._enter() May 19 11:57:42.079026 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 11:57:42.079026 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 11:57:42.079854 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 11:57:42.079854 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.creator() May 19 11:57:42.079854 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 11:57:42.079854 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application created_value = creator( May 19 11:57:42.079854 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 11:57:42.079854 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 11:57:42.079854 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 11:57:42.079854 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 11:57:42.079854 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 11:57:42.079854 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 11:57:42.079854 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 11:57:42.079854 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application May 19 11:57:42.082349 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 169/348] 10.222.0.23 () {56 vars in 1099 bytes} [Thu May 19 11:57:41 2022] GET /identity/v3/roles/member => generated 84 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 11:57:42.083333 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946cdbce-0fe1-4083-9bb2-b00014352ea0 None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:57:42.197403 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afdbc692-34c3-43e7-b338-61f14b77f3aa None None] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:57:42.232798 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946cdbce-0fe1-4083-9bb2-b00014352ea0 None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:57:42.240000 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e370a551-c6dc-4bc2-a109-3d46a7f6165c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:57:42.240663 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e370a551-c6dc-4bc2-a109-3d46a7f6165c None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:42.240951 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e370a551-c6dc-4bc2-a109-3d46a7f6165c None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:42.241231 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e370a551-c6dc-4bc2-a109-3d46a7f6165c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:42.280394 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d789a009-0e99-4ca4-911b-2283ba3a12df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:57:42.281091 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d789a009-0e99-4ca4-911b-2283ba3a12df None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:42.281348 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d789a009-0e99-4ca4-911b-2283ba3a12df None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:42.281623 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d789a009-0e99-4ca4-911b-2283ba3a12df None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:42.282684 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d789a009-0e99-4ca4-911b-2283ba3a12df None admin] RBAC: Adding query filter params (role.id=a49254f8009c463e88e5e0b3909c5904, user.id=07e60193e21e42f4bd442f0292b93267, scope.system=all) {{(pid=80539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 11:57:42.282805 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d789a009-0e99-4ca4-911b-2283ba3a12df None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:57:42.284339 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d789a009-0e99-4ca4-911b-2283ba3a12df None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:57:42.307816 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 170/349] 10.222.0.23 () {56 vars in 1308 bytes} [Thu May 19 11:57:42 2022] GET /identity/v3/role_assignments?user.id=07e60193e21e42f4bd442f0292b93267&scope.system=all&role.id=a49254f8009c463e88e5e0b3909c5904 => generated 510 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:57:42.307816 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d789a009-0e99-4ca4-911b-2283ba3a12df None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:57:42.330819 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:57:42.345574 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e370a551-c6dc-4bc2-a109-3d46a7f6165c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:57:42.347482 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e370a551-c6dc-4bc2-a109-3d46a7f6165c None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:57:42.366948 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:57:42.459508 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d789a009-0e99-4ca4-911b-2283ba3a12df None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:57:42.492367 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 180/350] 10.222.0.23 () {58 vars in 1282 bytes} [Thu May 19 11:57:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 441 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 11:57:42.493585 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e370a551-c6dc-4bc2-a109-3d46a7f6165c None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:57:42.518509 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ea8835-88a7-4c0c-8a32-56ec043b7909 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:57:42.519201 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ea8835-88a7-4c0c-8a32-56ec043b7909 None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:42.519484 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ea8835-88a7-4c0c-8a32-56ec043b7909 None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:42.519761 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ea8835-88a7-4c0c-8a32-56ec043b7909 None admin] PATH_INFO: `/v3/roles` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:42.520571 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ea8835-88a7-4c0c-8a32-56ec043b7909 None admin] RBAC: Adding query filter params (name=member) {{(pid=80539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 11:57:42.520882 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ea8835-88a7-4c0c-8a32-56ec043b7909 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:57:42.523074 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ea8835-88a7-4c0c-8a32-56ec043b7909 None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:57:42.538671 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 171/351] 10.222.0.23 () {56 vars in 1101 bytes} [Thu May 19 11:57:42 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:57:42.538671 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317369ce-27e2-4bac-9e07-325e551861d2 None None] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:42.538671 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317369ce-27e2-4bac-9e07-325e551861d2 None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:42.538671 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317369ce-27e2-4bac-9e07-325e551861d2 None None] PATH_INFO: `/` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:42.539915 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 172/352] 10.222.0.23 () {54 vars in 926 bytes} [Thu May 19 11:57:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 11:57:42.540639 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317369ce-27e2-4bac-9e07-325e551861d2 None None] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:57:42.629122 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:57:42.639695 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e370a551-c6dc-4bc2-a109-3d46a7f6165c None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:57:42.674377 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:57:42.677846 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317369ce-27e2-4bac-9e07-325e551861d2 None None] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:57:42.683217 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15c16b7-09af-4a92-bed7-6cfefac62beb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:57:42.684068 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15c16b7-09af-4a92-bed7-6cfefac62beb None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:42.684357 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15c16b7-09af-4a92-bed7-6cfefac62beb None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:42.684640 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15c16b7-09af-4a92-bed7-6cfefac62beb None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:42.699384 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15c16b7-09af-4a92-bed7-6cfefac62beb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 11:57:42.708425 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application [None req-b15c16b7-09af-4a92-bed7-6cfefac62beb None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 11:57:42.708425 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 11:57:42.708425 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 11:57:42.708425 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 11:57:42.708425 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 11:57:42.708425 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 11:57:42.708425 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 11:57:42.708425 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 11:57:42.708425 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 11:57:42.708425 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 11:57:42.708425 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 11:57:42.708425 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 11:57:42.708425 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 11:57:42.708425 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 11:57:42.708425 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 11:57:42.708425 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 11:57:42.709313 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 11:57:42.709313 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 11:57:42.709313 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 11:57:42.709313 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 11:57:42.709313 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 11:57:42.709313 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 11:57:42.709313 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 11:57:42.709313 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 11:57:42.709313 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 11:57:42.709313 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.get_or_create( May 19 11:57:42.709313 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 11:57:42.709313 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application with Lock( May 19 11:57:42.709313 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 11:57:42.709313 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._enter() May 19 11:57:42.709313 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 11:57:42.709313 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 11:57:42.710112 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 11:57:42.710112 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.creator() May 19 11:57:42.710112 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 11:57:42.710112 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application created_value = creator( May 19 11:57:42.710112 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 11:57:42.710112 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 11:57:42.710112 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 11:57:42.710112 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 11:57:42.710112 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 11:57:42.710112 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 11:57:42.710112 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 11:57:42.710112 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application May 19 11:57:42.710112 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 181/353] 10.222.0.23 () {56 vars in 1114 bytes} [Thu May 19 11:57:42 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 11:57:42.710774 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15c16b7-09af-4a92-bed7-6cfefac62beb None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:57:42.724958 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c030d3b-afe1-4215-b1c6-618469cc1909 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:57:42.726277 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c030d3b-afe1-4215-b1c6-618469cc1909 None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:42.726703 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c030d3b-afe1-4215-b1c6-618469cc1909 None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:42.727003 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c030d3b-afe1-4215-b1c6-618469cc1909 None admin] PATH_INFO: `/v3/projects/0c5d68529f48460f9f6e03e3f8fb205b` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:42.728593 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c030d3b-afe1-4215-b1c6-618469cc1909 None admin] RBAC: Authorizing `identity:get_project(project_id=0c5d68529f48460f9f6e03e3f8fb205b)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:57:42.730613 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c030d3b-afe1-4215-b1c6-618469cc1909 None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:57:42.732738 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 173/354] 10.222.0.23 () {56 vars in 1186 bytes} [Thu May 19 11:57:42 2022] GET /identity/v3/projects/0c5d68529f48460f9f6e03e3f8fb205b => generated 302 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:57:42.736837 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b71993-69d4-4333-a3cf-2e5978891adf None None] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:42.736837 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b71993-69d4-4333-a3cf-2e5978891adf None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:42.736837 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b71993-69d4-4333-a3cf-2e5978891adf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:42.753054 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.auth.core [None req-25b71993-69d4-4333-a3cf-2e5978891adf None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 11:57:42.754289 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-25b71993-69d4-4333-a3cf-2e5978891adf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 11:57:42.838299 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15c16b7-09af-4a92-bed7-6cfefac62beb None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:57:42.845442 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 174/355] 10.222.0.23 () {58 vars in 1023 bytes} [Thu May 19 11:57:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 11:57:42.846408 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b71993-69d4-4333-a3cf-2e5978891adf None None] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:57:42.886804 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff6ab90-94cf-4d3b-b98f-dcacfac3e77e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:57:42.892083 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff6ab90-94cf-4d3b-b98f-dcacfac3e77e None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:42.892083 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff6ab90-94cf-4d3b-b98f-dcacfac3e77e None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:42.892083 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff6ab90-94cf-4d3b-b98f-dcacfac3e77e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:42.913338 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:57:42.976995 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff6ab90-94cf-4d3b-b98f-dcacfac3e77e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:57:42.978852 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff6ab90-94cf-4d3b-b98f-dcacfac3e77e None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:57:42.982208 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:57:42.986497 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b71993-69d4-4333-a3cf-2e5978891adf None None] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:57:43.035427 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6174511-7d94-45f3-ad78-ba8bcd10a0d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:57:43.036376 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6174511-7d94-45f3-ad78-ba8bcd10a0d2 None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:43.036484 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6174511-7d94-45f3-ad78-ba8bcd10a0d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:43.036727 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6174511-7d94-45f3-ad78-ba8bcd10a0d2 None admin] PATH_INFO: `/v3/roles` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:43.037807 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6174511-7d94-45f3-ad78-ba8bcd10a0d2 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 11:57:43.038105 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6174511-7d94-45f3-ad78-ba8bcd10a0d2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:57:43.040153 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6174511-7d94-45f3-ad78-ba8bcd10a0d2 None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:57:43.052410 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 175/356] 10.222.0.23 () {56 vars in 1111 bytes} [Thu May 19 11:57:42 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:57:43.053289 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6174511-7d94-45f3-ad78-ba8bcd10a0d2 None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:57:43.112164 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 182/357] 10.222.0.23 () {58 vars in 1282 bytes} [Thu May 19 11:57:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 11:57:43.113629 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff6ab90-94cf-4d3b-b98f-dcacfac3e77e None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:57:43.175518 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6174511-7d94-45f3-ad78-ba8bcd10a0d2 None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:57:43.179343 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:57:43.219488 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12555360-00e4-4345-9516-f6e472c8b611 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:57:43.220404 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12555360-00e4-4345-9516-f6e472c8b611 None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:43.220687 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12555360-00e4-4345-9516-f6e472c8b611 None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:43.220963 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12555360-00e4-4345-9516-f6e472c8b611 None admin] PATH_INFO: `/v3/groups/5f71f3c2dd1d47f980fee625c78a2dee` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:43.227476 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12555360-00e4-4345-9516-f6e472c8b611 None admin] RBAC: Authorizing `identity:get_group(group_id=5f71f3c2dd1d47f980fee625c78a2dee)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:57:43.229183 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12555360-00e4-4345-9516-f6e472c8b611 None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:57:43.230892 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 176/358] 10.222.0.23 () {56 vars in 1180 bytes} [Thu May 19 11:57:43 2022] GET /identity/v3/groups/5f71f3c2dd1d47f980fee625c78a2dee => generated 226 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:57:43.235730 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e393a7ad-6d2c-4c4d-a4fe-94bb1d877bd8 None None] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:43.236051 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e393a7ad-6d2c-4c4d-a4fe-94bb1d877bd8 None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:43.236336 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e393a7ad-6d2c-4c4d-a4fe-94bb1d877bd8 None None] PATH_INFO: `/` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:43.240760 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 177/359] 10.222.0.23 () {54 vars in 926 bytes} [Thu May 19 11:57:43 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 11:57:43.240760 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e393a7ad-6d2c-4c4d-a4fe-94bb1d877bd8 None None] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:57:43.242208 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff6ab90-94cf-4d3b-b98f-dcacfac3e77e None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:57:43.270829 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:57:43.284316 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faed429b-7941-4bdf-8ad8-edb7fc7a958c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:57:43.287412 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faed429b-7941-4bdf-8ad8-edb7fc7a958c None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:43.287412 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faed429b-7941-4bdf-8ad8-edb7fc7a958c None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:43.287412 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faed429b-7941-4bdf-8ad8-edb7fc7a958c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:43.299366 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faed429b-7941-4bdf-8ad8-edb7fc7a958c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 11:57:43.306869 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application [None req-faed429b-7941-4bdf-8ad8-edb7fc7a958c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 11:57:43.306869 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 11:57:43.306869 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 11:57:43.306869 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 11:57:43.306869 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 11:57:43.306869 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 11:57:43.306869 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 11:57:43.306869 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 11:57:43.306869 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 11:57:43.306869 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 11:57:43.306869 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 11:57:43.306869 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 11:57:43.306869 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 11:57:43.306869 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 11:57:43.306869 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 11:57:43.306869 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 11:57:43.307753 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 11:57:43.307753 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 11:57:43.307753 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 11:57:43.307753 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 11:57:43.307753 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 11:57:43.307753 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 11:57:43.307753 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 11:57:43.307753 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 11:57:43.307753 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 11:57:43.307753 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.get_or_create( May 19 11:57:43.307753 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 11:57:43.307753 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application with Lock( May 19 11:57:43.307753 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 11:57:43.307753 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._enter() May 19 11:57:43.307753 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 11:57:43.307753 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 11:57:43.308578 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 11:57:43.308578 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.creator() May 19 11:57:43.308578 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 11:57:43.308578 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application created_value = creator( May 19 11:57:43.308578 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 11:57:43.308578 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 11:57:43.308578 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 11:57:43.308578 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 11:57:43.308578 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 11:57:43.308578 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 11:57:43.308578 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 11:57:43.308578 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application May 19 11:57:43.308578 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 183/360] 10.222.0.23 () {56 vars in 1114 bytes} [Thu May 19 11:57:43 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 11:57:43.308578 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faed429b-7941-4bdf-8ad8-edb7fc7a958c None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:57:43.364018 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e393a7ad-6d2c-4c4d-a4fe-94bb1d877bd8 None None] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:57:43.404427 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ae85a1-cf16-452a-a7de-e7d4a887d443 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:57:43.405106 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ae85a1-cf16-452a-a7de-e7d4a887d443 None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:43.405390 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ae85a1-cf16-452a-a7de-e7d4a887d443 None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:43.405663 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ae85a1-cf16-452a-a7de-e7d4a887d443 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:43.437491 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faed429b-7941-4bdf-8ad8-edb7fc7a958c None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:57:43.457526 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:57:43.482624 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f52203-257b-4d75-b510-fb2edb98cf1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:57:43.484163 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f52203-257b-4d75-b510-fb2edb98cf1e None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:43.484453 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f52203-257b-4d75-b510-fb2edb98cf1e None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:43.484830 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f52203-257b-4d75-b510-fb2edb98cf1e None admin] PATH_INFO: `/v3/projects/64092131887b4859ae47a1f4c3e0d24a` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:43.486794 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f52203-257b-4d75-b510-fb2edb98cf1e None admin] RBAC: Authorizing `identity:get_project(project_id=64092131887b4859ae47a1f4c3e0d24a)` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:57:43.486907 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ae85a1-cf16-452a-a7de-e7d4a887d443 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:57:43.487667 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f52203-257b-4d75-b510-fb2edb98cf1e None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:57:43.487807 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ae85a1-cf16-452a-a7de-e7d4a887d443 None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:57:43.489870 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 184/361] 10.222.0.23 () {56 vars in 1186 bytes} [Thu May 19 11:57:43 2022] GET /identity/v3/projects/64092131887b4859ae47a1f4c3e0d24a => generated 298 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:57:43.491579 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f52203-257b-4d75-b510-fb2edb98cf1e None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:57:43.516764 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:57:43.604822 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 178/362] 10.222.0.23 () {58 vars in 1282 bytes} [Thu May 19 11:57:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 11:57:43.606483 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8167f5a-c65b-48da-97c4-6ec410b8f031 None None] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:43.606974 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8167f5a-c65b-48da-97c4-6ec410b8f031 None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:43.607478 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8167f5a-c65b-48da-97c4-6ec410b8f031 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:43.618318 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f52203-257b-4d75-b510-fb2edb98cf1e None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:57:43.626413 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.auth.core [None req-d8167f5a-c65b-48da-97c4-6ec410b8f031 None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 11:57:43.627902 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-d8167f5a-c65b-48da-97c4-6ec410b8f031 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 11:57:43.664846 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad79bc4-b407-4d6d-89da-d59ba1387776 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:57:43.665781 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad79bc4-b407-4d6d-89da-d59ba1387776 None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:43.666066 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad79bc4-b407-4d6d-89da-d59ba1387776 None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:43.666344 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad79bc4-b407-4d6d-89da-d59ba1387776 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:43.667251 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad79bc4-b407-4d6d-89da-d59ba1387776 None admin] RBAC: Adding query filter params (group.id=5f71f3c2dd1d47f980fee625c78a2dee, role.id=259eb400cdb4439db72d14456ae9cd08, scope.project.id=0c5d68529f48460f9f6e03e3f8fb205b) {{(pid=80540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 11:57:43.667494 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad79bc4-b407-4d6d-89da-d59ba1387776 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:57:43.668990 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad79bc4-b407-4d6d-89da-d59ba1387776 None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:57:43.676212 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 185/363] 10.222.0.23 () {56 vars in 1376 bytes} [Thu May 19 11:57:43 2022] GET /identity/v3/role_assignments?group.id=5f71f3c2dd1d47f980fee625c78a2dee&scope.project.id=0c5d68529f48460f9f6e03e3f8fb205b&role.id=259eb400cdb4439db72d14456ae9cd08 => generated 260 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:57:43.677161 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad79bc4-b407-4d6d-89da-d59ba1387776 None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:57:43.721875 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 179/364] 10.222.0.23 () {58 vars in 1023 bytes} [Thu May 19 11:57:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 11:57:43.723229 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8167f5a-c65b-48da-97c4-6ec410b8f031 None None] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:57:43.739251 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:57:43.812874 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:57:43.825459 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad79bc4-b407-4d6d-89da-d59ba1387776 None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:57:43.871838 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c80128-8a6b-4674-b268-903f2e0236bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:57:43.872563 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c80128-8a6b-4674-b268-903f2e0236bc None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:43.872846 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c80128-8a6b-4674-b268-903f2e0236bc None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:43.873121 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c80128-8a6b-4674-b268-903f2e0236bc None admin] PATH_INFO: `/v3/roles` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:43.874388 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c80128-8a6b-4674-b268-903f2e0236bc None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 11:57:43.874703 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c80128-8a6b-4674-b268-903f2e0236bc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:57:43.876173 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c80128-8a6b-4674-b268-903f2e0236bc None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:57:43.877117 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8167f5a-c65b-48da-97c4-6ec410b8f031 None None] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:57:43.884836 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 186/365] 10.222.0.23 () {56 vars in 1111 bytes} [Thu May 19 11:57:43 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:57:43.885738 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c80128-8a6b-4674-b268-903f2e0236bc None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:57:43.919583 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6821c670-8adc-4b86-8dc7-c21cdb3c352a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:57:43.920563 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6821c670-8adc-4b86-8dc7-c21cdb3c352a None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:43.921717 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6821c670-8adc-4b86-8dc7-c21cdb3c352a None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:43.921997 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6821c670-8adc-4b86-8dc7-c21cdb3c352a None admin] PATH_INFO: `/v3/groups/5f71f3c2dd1d47f980fee625c78a2dee` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:43.923418 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6821c670-8adc-4b86-8dc7-c21cdb3c352a None admin] RBAC: Authorizing `identity:get_group(group_id=5f71f3c2dd1d47f980fee625c78a2dee)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:57:43.925004 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6821c670-8adc-4b86-8dc7-c21cdb3c352a None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:57:43.927139 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 180/366] 10.222.0.23 () {56 vars in 1180 bytes} [Thu May 19 11:57:43 2022] GET /identity/v3/groups/5f71f3c2dd1d47f980fee625c78a2dee => generated 226 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:57:43.928002 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6821c670-8adc-4b86-8dc7-c21cdb3c352a None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:57:44.004476 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c80128-8a6b-4674-b268-903f2e0236bc None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:57:44.012694 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:57:44.052952 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fcb39d-0d4d-4218-9b35-1192869babc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:57:44.054295 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fcb39d-0d4d-4218-9b35-1192869babc3 None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:44.054655 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fcb39d-0d4d-4218-9b35-1192869babc3 None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:44.055214 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fcb39d-0d4d-4218-9b35-1192869babc3 None admin] PATH_INFO: `/v3/projects/64092131887b4859ae47a1f4c3e0d24a` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:44.056995 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7fcb39d-0d4d-4218-9b35-1192869babc3 None admin] RBAC: Authorizing `identity:get_project(project_id=64092131887b4859ae47a1f4c3e0d24a)` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:57:44.058569 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7fcb39d-0d4d-4218-9b35-1192869babc3 None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:57:44.061142 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 187/367] 10.222.0.23 () {56 vars in 1186 bytes} [Thu May 19 11:57:43 2022] GET /identity/v3/projects/64092131887b4859ae47a1f4c3e0d24a => generated 298 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:57:44.062470 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fcb39d-0d4d-4218-9b35-1192869babc3 None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:57:44.081584 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6821c670-8adc-4b86-8dc7-c21cdb3c352a None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:57:44.113563 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:57:44.129252 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff3184c-88c3-472b-88ba-3b12ce1f2438 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:57:44.130336 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff3184c-88c3-472b-88ba-3b12ce1f2438 None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:44.130629 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff3184c-88c3-472b-88ba-3b12ce1f2438 None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:44.130908 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff3184c-88c3-472b-88ba-3b12ce1f2438 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:44.144643 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff3184c-88c3-472b-88ba-3b12ce1f2438 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 11:57:44.153692 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application [None req-6ff3184c-88c3-472b-88ba-3b12ce1f2438 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 11:57:44.153692 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 11:57:44.153692 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 11:57:44.153692 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 11:57:44.153692 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 11:57:44.153692 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 11:57:44.153692 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 11:57:44.153692 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 11:57:44.153692 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 11:57:44.153692 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 11:57:44.153692 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 11:57:44.153692 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 11:57:44.153692 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 11:57:44.153692 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 11:57:44.153692 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 11:57:44.153692 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 11:57:44.154631 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 11:57:44.154631 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 11:57:44.154631 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 11:57:44.154631 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 11:57:44.154631 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 11:57:44.154631 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 11:57:44.154631 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 11:57:44.154631 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 11:57:44.154631 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 11:57:44.154631 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.get_or_create( May 19 11:57:44.154631 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 11:57:44.154631 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application with Lock( May 19 11:57:44.154631 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 11:57:44.154631 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._enter() May 19 11:57:44.154631 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 11:57:44.154631 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 11:57:44.155509 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 11:57:44.155509 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.creator() May 19 11:57:44.155509 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 11:57:44.155509 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application created_value = creator( May 19 11:57:44.155509 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 11:57:44.155509 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 11:57:44.155509 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 11:57:44.155509 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 11:57:44.155509 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 11:57:44.155509 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 11:57:44.155509 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 11:57:44.155509 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application May 19 11:57:44.155509 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 181/368] 10.222.0.23 () {56 vars in 1096 bytes} [Thu May 19 11:57:43 2022] GET /identity/v3/roles/admin => generated 83 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 11:57:44.156988 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff3184c-88c3-472b-88ba-3b12ce1f2438 None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:57:44.189351 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fcb39d-0d4d-4218-9b35-1192869babc3 None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:57:44.241800 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7fac9b-7715-4fa6-bfe8-9d662e1bbd77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:57:44.242479 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7fac9b-7715-4fa6-bfe8-9d662e1bbd77 None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:44.242989 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7fac9b-7715-4fa6-bfe8-9d662e1bbd77 None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:44.243274 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7fac9b-7715-4fa6-bfe8-9d662e1bbd77 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:44.301452 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff3184c-88c3-472b-88ba-3b12ce1f2438 None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:57:44.303138 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:57:44.340603 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7fac9b-7715-4fa6-bfe8-9d662e1bbd77 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:57:44.342211 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7fac9b-7715-4fa6-bfe8-9d662e1bbd77 None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:57:44.347272 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4943f20-e5c5-440f-af53-b288cf0c06ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:57:44.349934 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4943f20-e5c5-440f-af53-b288cf0c06ee None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:44.349934 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4943f20-e5c5-440f-af53-b288cf0c06ee None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:44.349934 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4943f20-e5c5-440f-af53-b288cf0c06ee None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:44.349934 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4943f20-e5c5-440f-af53-b288cf0c06ee None admin] RBAC: Adding query filter params (group.id=5f71f3c2dd1d47f980fee625c78a2dee, role.id=259eb400cdb4439db72d14456ae9cd08, scope.project.id=64092131887b4859ae47a1f4c3e0d24a) {{(pid=80539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 11:57:44.349934 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4943f20-e5c5-440f-af53-b288cf0c06ee None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:57:44.351475 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4943f20-e5c5-440f-af53-b288cf0c06ee None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:57:44.361998 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 182/369] 10.222.0.23 () {56 vars in 1376 bytes} [Thu May 19 11:57:44 2022] GET /identity/v3/role_assignments?group.id=5f71f3c2dd1d47f980fee625c78a2dee&scope.project.id=64092131887b4859ae47a1f4c3e0d24a&role.id=259eb400cdb4439db72d14456ae9cd08 => generated 260 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:57:44.362767 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4943f20-e5c5-440f-af53-b288cf0c06ee None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:57:44.442873 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:57:44.494434 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 188/370] 10.222.0.23 () {58 vars in 1282 bytes} [Thu May 19 11:57:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 433 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 11:57:44.495817 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7fac9b-7715-4fa6-bfe8-9d662e1bbd77 None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:57:44.534689 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4943f20-e5c5-440f-af53-b288cf0c06ee None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:57:44.574541 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98024b9c-30b0-4405-9030-315942c2b398 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:57:44.575592 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98024b9c-30b0-4405-9030-315942c2b398 None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:44.575905 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98024b9c-30b0-4405-9030-315942c2b398 None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:44.576184 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98024b9c-30b0-4405-9030-315942c2b398 None admin] PATH_INFO: `/v3/groups/5f71f3c2dd1d47f980fee625c78a2dee` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:44.578004 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98024b9c-30b0-4405-9030-315942c2b398 None admin] RBAC: Authorizing `identity:get_group(group_id=5f71f3c2dd1d47f980fee625c78a2dee)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:57:44.579472 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98024b9c-30b0-4405-9030-315942c2b398 None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:57:44.583290 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 183/371] 10.222.0.23 () {56 vars in 1180 bytes} [Thu May 19 11:57:44 2022] GET /identity/v3/groups/5f71f3c2dd1d47f980fee625c78a2dee => generated 226 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:57:44.584144 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98024b9c-30b0-4405-9030-315942c2b398 None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:57:44.613264 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:57:44.644050 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7fac9b-7715-4fa6-bfe8-9d662e1bbd77 None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:57:44.699936 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447a27de-0d13-4599-a9b1-73e05f96a7ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:57:44.700537 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447a27de-0d13-4599-a9b1-73e05f96a7ac None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:44.700827 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447a27de-0d13-4599-a9b1-73e05f96a7ac None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:44.701102 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447a27de-0d13-4599-a9b1-73e05f96a7ac None admin] PATH_INFO: `/v3/roles` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:44.701864 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447a27de-0d13-4599-a9b1-73e05f96a7ac None admin] RBAC: Adding query filter params (name=admin) {{(pid=80540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 11:57:44.702956 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447a27de-0d13-4599-a9b1-73e05f96a7ac None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:57:44.704440 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447a27de-0d13-4599-a9b1-73e05f96a7ac None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:57:44.719300 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 189/372] 10.222.0.23 () {56 vars in 1099 bytes} [Thu May 19 11:57:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:57:44.720824 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447a27de-0d13-4599-a9b1-73e05f96a7ac None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:57:44.732823 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98024b9c-30b0-4405-9030-315942c2b398 None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:57:44.754662 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:57:44.778408 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd122d1-3fe3-4932-94f1-ec2a515566d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:57:44.779342 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd122d1-3fe3-4932-94f1-ec2a515566d3 None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:44.779660 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd122d1-3fe3-4932-94f1-ec2a515566d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:44.781094 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd122d1-3fe3-4932-94f1-ec2a515566d3 None admin] PATH_INFO: `/v3/projects/0c5d68529f48460f9f6e03e3f8fb205b` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:44.782982 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd122d1-3fe3-4932-94f1-ec2a515566d3 None admin] RBAC: Authorizing `identity:get_project(project_id=0c5d68529f48460f9f6e03e3f8fb205b)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:57:44.784660 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd122d1-3fe3-4932-94f1-ec2a515566d3 None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:57:44.791775 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 184/373] 10.222.0.23 () {56 vars in 1186 bytes} [Thu May 19 11:57:44 2022] GET /identity/v3/projects/0c5d68529f48460f9f6e03e3f8fb205b => generated 302 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:57:44.792663 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd122d1-3fe3-4932-94f1-ec2a515566d3 None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:57:44.855600 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447a27de-0d13-4599-a9b1-73e05f96a7ac None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:57:44.893009 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea76edf-e045-436a-b589-188ea3a98576 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:57:44.893706 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea76edf-e045-436a-b589-188ea3a98576 None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:44.893994 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea76edf-e045-436a-b589-188ea3a98576 None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:44.894276 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea76edf-e045-436a-b589-188ea3a98576 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:44.895109 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea76edf-e045-436a-b589-188ea3a98576 None admin] RBAC: Adding query filter params (group.id=5f71f3c2dd1d47f980fee625c78a2dee, role.id=8215583bb4224c75b093ae8e3993eb9f, scope.project.id=64092131887b4859ae47a1f4c3e0d24a) {{(pid=80540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 11:57:44.895425 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea76edf-e045-436a-b589-188ea3a98576 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:57:44.896911 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea76edf-e045-436a-b589-188ea3a98576 None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:57:44.899562 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd122d1-3fe3-4932-94f1-ec2a515566d3 None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:57:44.899995 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:57:44.902893 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 190/374] 10.222.0.23 () {56 vars in 1376 bytes} [Thu May 19 11:57:44 2022] GET /identity/v3/role_assignments?group.id=5f71f3c2dd1d47f980fee625c78a2dee&scope.project.id=64092131887b4859ae47a1f4c3e0d24a&role.id=8215583bb4224c75b093ae8e3993eb9f => generated 260 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:57:44.903811 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea76edf-e045-436a-b589-188ea3a98576 None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:57:44.946848 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab0b773-cb9b-488c-9a82-68f6e71e445d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:57:44.946848 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab0b773-cb9b-488c-9a82-68f6e71e445d None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:44.946848 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab0b773-cb9b-488c-9a82-68f6e71e445d None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:44.946848 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab0b773-cb9b-488c-9a82-68f6e71e445d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:45.025061 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea76edf-e045-436a-b589-188ea3a98576 None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:57:45.040161 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:57:45.070648 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025ecb74-7f6e-4f3f-afc0-c5d3163c7e63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:57:45.072194 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025ecb74-7f6e-4f3f-afc0-c5d3163c7e63 None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:45.072933 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025ecb74-7f6e-4f3f-afc0-c5d3163c7e63 None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:45.073653 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025ecb74-7f6e-4f3f-afc0-c5d3163c7e63 None admin] PATH_INFO: `/v3/groups/5f71f3c2dd1d47f980fee625c78a2dee` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:45.075623 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025ecb74-7f6e-4f3f-afc0-c5d3163c7e63 None admin] RBAC: Authorizing `identity:get_group(group_id=5f71f3c2dd1d47f980fee625c78a2dee)` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:57:45.077779 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025ecb74-7f6e-4f3f-afc0-c5d3163c7e63 None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:57:45.080671 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 191/375] 10.222.0.23 () {56 vars in 1180 bytes} [Thu May 19 11:57:44 2022] GET /identity/v3/groups/5f71f3c2dd1d47f980fee625c78a2dee => generated 226 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:57:45.092352 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab0b773-cb9b-488c-9a82-68f6e71e445d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:57:45.094666 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab0b773-cb9b-488c-9a82-68f6e71e445d None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:57:45.095492 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025ecb74-7f6e-4f3f-afc0-c5d3163c7e63 None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:57:45.207536 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:57:45.245048 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 185/376] 10.222.0.23 () {58 vars in 1282 bytes} [Thu May 19 11:57:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 453 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 11:57:45.251057 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab0b773-cb9b-488c-9a82-68f6e71e445d None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:57:45.270933 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025ecb74-7f6e-4f3f-afc0-c5d3163c7e63 None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:57:45.307719 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e233c2-b86a-47eb-bc3a-38ad5c43ffc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:57:45.308410 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e233c2-b86a-47eb-bc3a-38ad5c43ffc5 None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:45.309115 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e233c2-b86a-47eb-bc3a-38ad5c43ffc5 None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:45.309397 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e233c2-b86a-47eb-bc3a-38ad5c43ffc5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:45.310384 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e233c2-b86a-47eb-bc3a-38ad5c43ffc5 None admin] RBAC: Adding query filter params (group.id=5f71f3c2dd1d47f980fee625c78a2dee, role.id=8215583bb4224c75b093ae8e3993eb9f, scope.project.id=0c5d68529f48460f9f6e03e3f8fb205b) {{(pid=80540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 11:57:45.310700 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e233c2-b86a-47eb-bc3a-38ad5c43ffc5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:57:45.312750 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e233c2-b86a-47eb-bc3a-38ad5c43ffc5 None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:57:45.319295 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 192/377] 10.222.0.23 () {56 vars in 1376 bytes} [Thu May 19 11:57:45 2022] GET /identity/v3/role_assignments?group.id=5f71f3c2dd1d47f980fee625c78a2dee&scope.project.id=0c5d68529f48460f9f6e03e3f8fb205b&role.id=8215583bb4224c75b093ae8e3993eb9f => generated 260 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 11:57:45.376427 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:57:45.396803 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab0b773-cb9b-488c-9a82-68f6e71e445d None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:57:45.433347 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35740872-6c40-41ec-b0dd-34fed1aa3d1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:57:45.434243 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35740872-6c40-41ec-b0dd-34fed1aa3d1a None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:45.434524 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35740872-6c40-41ec-b0dd-34fed1aa3d1a None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:45.434799 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35740872-6c40-41ec-b0dd-34fed1aa3d1a None admin] PATH_INFO: `/v3/projects/f3569780f2fe4b24b4109e2eee3c26fe` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:45.436162 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35740872-6c40-41ec-b0dd-34fed1aa3d1a None admin] RBAC: Authorizing `identity:get_project(project_id=f3569780f2fe4b24b4109e2eee3c26fe)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:57:45.438211 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35740872-6c40-41ec-b0dd-34fed1aa3d1a None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:57:45.440464 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 186/378] 10.222.0.23 () {56 vars in 1186 bytes} [Thu May 19 11:57:45 2022] GET /identity/v3/projects/f3569780f2fe4b24b4109e2eee3c26fe => generated 344 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 11:57:45.446328 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e233c2-b86a-47eb-bc3a-38ad5c43ffc5 None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:57:45.586311 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e233c2-b86a-47eb-bc3a-38ad5c43ffc5 None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:57:45.617654 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:57:45.628597 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb0d567-2f2a-40fc-a55a-4821da4a5a3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:57:45.629486 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb0d567-2f2a-40fc-a55a-4821da4a5a3e None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:45.632615 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb0d567-2f2a-40fc-a55a-4821da4a5a3e None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:45.632615 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb0d567-2f2a-40fc-a55a-4821da4a5a3e None admin] PATH_INFO: `/v3/groups/2b83c34e4dc54b2fb1afe8a56166cd2c` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:45.641359 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb0d567-2f2a-40fc-a55a-4821da4a5a3e None admin] RBAC: Authorizing `identity:get_group(group_id=2b83c34e4dc54b2fb1afe8a56166cd2c)` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:57:45.642840 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb0d567-2f2a-40fc-a55a-4821da4a5a3e None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:57:45.644895 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 193/379] 10.222.0.23 () {56 vars in 1180 bytes} [Thu May 19 11:57:45 2022] GET /identity/v3/groups/2b83c34e4dc54b2fb1afe8a56166cd2c => generated 229 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:57:45.650627 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35740872-6c40-41ec-b0dd-34fed1aa3d1a None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:57:45.798911 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35740872-6c40-41ec-b0dd-34fed1aa3d1a None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:57:45.836859 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f7976b-dfdf-4500-b05d-95cfee30af17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:57:45.837937 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f7976b-dfdf-4500-b05d-95cfee30af17 None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:45.838473 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f7976b-dfdf-4500-b05d-95cfee30af17 None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:45.838985 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f7976b-dfdf-4500-b05d-95cfee30af17 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:45.840146 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f7976b-dfdf-4500-b05d-95cfee30af17 None admin] RBAC: Adding query filter params (group.id=2b83c34e4dc54b2fb1afe8a56166cd2c, role.id=9db5671c65d3414ea2682afa2ccf1403, scope.project.id=f3569780f2fe4b24b4109e2eee3c26fe) {{(pid=80539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 11:57:45.840731 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f7976b-dfdf-4500-b05d-95cfee30af17 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:57:45.842526 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f7976b-dfdf-4500-b05d-95cfee30af17 None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:57:45.850064 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 187/380] 10.222.0.23 () {56 vars in 1376 bytes} [Thu May 19 11:57:45 2022] GET /identity/v3/role_assignments?group.id=2b83c34e4dc54b2fb1afe8a56166cd2c&scope.project.id=f3569780f2fe4b24b4109e2eee3c26fe&role.id=9db5671c65d3414ea2682afa2ccf1403 => generated 260 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:57:46.266306 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f90fad6-4d54-4c16-9daa-f54f179f9b2c None None] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:46.266788 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f90fad6-4d54-4c16-9daa-f54f179f9b2c None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:46.266861 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f90fad6-4d54-4c16-9daa-f54f179f9b2c None None] PATH_INFO: `/` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:46.268605 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 194/381] 10.222.0.23 () {54 vars in 926 bytes} [Thu May 19 11:57:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 11:57:46.274251 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c497710c-bf8c-4376-81ac-1d4d68ddcf88 None None] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:46.274658 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c497710c-bf8c-4376-81ac-1d4d68ddcf88 None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:46.274841 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c497710c-bf8c-4376-81ac-1d4d68ddcf88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:46.292852 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.auth.core [None req-c497710c-bf8c-4376-81ac-1d4d68ddcf88 None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 11:57:46.294066 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-c497710c-bf8c-4376-81ac-1d4d68ddcf88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 11:57:46.307446 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:57:46.398509 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 188/382] 10.222.0.23 () {58 vars in 1023 bytes} [Thu May 19 11:57:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 11:57:46.433272 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f90fad6-4d54-4c16-9daa-f54f179f9b2c None None] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:57:46.565528 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f90fad6-4d54-4c16-9daa-f54f179f9b2c None None] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:57:46.616545 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a1e82d-16e0-49e0-8861-26bf6d0e4290 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:57:46.617355 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a1e82d-16e0-49e0-8861-26bf6d0e4290 None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:46.617643 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a1e82d-16e0-49e0-8861-26bf6d0e4290 None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:46.617920 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a1e82d-16e0-49e0-8861-26bf6d0e4290 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:46.633474 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a1e82d-16e0-49e0-8861-26bf6d0e4290 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 11:57:46.641361 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application [None req-48a1e82d-16e0-49e0-8861-26bf6d0e4290 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 11:57:46.641361 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 11:57:46.641361 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 11:57:46.641361 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 11:57:46.641361 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 11:57:46.641361 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 11:57:46.641361 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 11:57:46.641361 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 11:57:46.641361 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 11:57:46.641361 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 11:57:46.641361 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 11:57:46.641361 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 11:57:46.641361 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 11:57:46.641361 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 11:57:46.641361 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 11:57:46.641361 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 11:57:46.642274 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 11:57:46.642274 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 11:57:46.642274 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 11:57:46.642274 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 11:57:46.642274 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 11:57:46.642274 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 11:57:46.642274 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 11:57:46.642274 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 11:57:46.642274 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 11:57:46.642274 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.get_or_create( May 19 11:57:46.642274 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 11:57:46.642274 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application with Lock( May 19 11:57:46.642274 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 11:57:46.642274 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._enter() May 19 11:57:46.642274 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 11:57:46.642274 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 11:57:46.643033 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 11:57:46.643033 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.creator() May 19 11:57:46.643033 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 11:57:46.643033 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application created_value = creator( May 19 11:57:46.643033 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 11:57:46.643033 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 11:57:46.643033 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 11:57:46.643033 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 11:57:46.643033 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 11:57:46.643033 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 11:57:46.643033 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 11:57:46.643033 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application May 19 11:57:46.647033 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 195/383] 10.222.0.23 () {56 vars in 1099 bytes} [Thu May 19 11:57:46 2022] GET /identity/v3/roles/member => generated 84 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 11:57:46.648412 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c497710c-bf8c-4376-81ac-1d4d68ddcf88 None None] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:57:46.802625 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c497710c-bf8c-4376-81ac-1d4d68ddcf88 None None] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:57:46.839487 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6374cdf-5509-40d8-9b4b-e821e0442ab3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:57:46.840117 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6374cdf-5509-40d8-9b4b-e821e0442ab3 None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:46.840406 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6374cdf-5509-40d8-9b4b-e821e0442ab3 None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:46.840680 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6374cdf-5509-40d8-9b4b-e821e0442ab3 None admin] PATH_INFO: `/v3/roles` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:46.841514 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6374cdf-5509-40d8-9b4b-e821e0442ab3 None admin] RBAC: Adding query filter params (name=member) {{(pid=80539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 11:57:46.841818 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6374cdf-5509-40d8-9b4b-e821e0442ab3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:57:46.843347 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6374cdf-5509-40d8-9b4b-e821e0442ab3 None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:57:46.852732 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 189/384] 10.222.0.23 () {56 vars in 1101 bytes} [Thu May 19 11:57:46 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:57:46.858721 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a1e82d-16e0-49e0-8861-26bf6d0e4290 None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:57:46.922613 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:57:46.982835 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a1e82d-16e0-49e0-8861-26bf6d0e4290 None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:57:47.028890 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5cc28b6-d162-4eb9-906a-6f98e75e6edc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:57:47.029812 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cc28b6-d162-4eb9-906a-6f98e75e6edc None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:47.030744 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cc28b6-d162-4eb9-906a-6f98e75e6edc None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:47.031026 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cc28b6-d162-4eb9-906a-6f98e75e6edc None admin] PATH_INFO: `/v3/groups/5f71f3c2dd1d47f980fee625c78a2dee` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:47.036518 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5cc28b6-d162-4eb9-906a-6f98e75e6edc None admin] RBAC: Authorizing `identity:get_group(group_id=5f71f3c2dd1d47f980fee625c78a2dee)` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:57:47.038010 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5cc28b6-d162-4eb9-906a-6f98e75e6edc None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:57:47.039886 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 196/385] 10.222.0.23 () {56 vars in 1180 bytes} [Thu May 19 11:57:46 2022] GET /identity/v3/groups/5f71f3c2dd1d47f980fee625c78a2dee => generated 226 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:57:47.045190 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6374cdf-5509-40d8-9b4b-e821e0442ab3 None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:57:47.058367 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:57:47.166431 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5303fc-5306-4dab-aa85-64c3685d0257 None None] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:47.167060 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5303fc-5306-4dab-aa85-64c3685d0257 None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:47.167621 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5303fc-5306-4dab-aa85-64c3685d0257 None None] PATH_INFO: `/` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:47.169466 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 197/386] 10.222.0.23 () {54 vars in 926 bytes} [Thu May 19 11:57:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 11:57:47.176020 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2892ebff-6f3b-420a-a16b-0a3a4760f7d4 None None] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:47.176712 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2892ebff-6f3b-420a-a16b-0a3a4760f7d4 None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:47.177187 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2892ebff-6f3b-420a-a16b-0a3a4760f7d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:47.197640 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.auth.core [None req-2892ebff-6f3b-420a-a16b-0a3a4760f7d4 None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 11:57:47.199710 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-2892ebff-6f3b-420a-a16b-0a3a4760f7d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 11:57:47.210159 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6374cdf-5509-40d8-9b4b-e821e0442ab3 None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:57:47.277959 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97708813-f0b2-499b-9e69-eacb168c878c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:57:47.278834 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97708813-f0b2-499b-9e69-eacb168c878c None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:47.279118 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97708813-f0b2-499b-9e69-eacb168c878c None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:47.279398 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97708813-f0b2-499b-9e69-eacb168c878c None admin] PATH_INFO: `/v3/projects/0c5d68529f48460f9f6e03e3f8fb205b` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:47.281988 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97708813-f0b2-499b-9e69-eacb168c878c None admin] RBAC: Authorizing `identity:get_project(project_id=0c5d68529f48460f9f6e03e3f8fb205b)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:57:47.283447 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97708813-f0b2-499b-9e69-eacb168c878c None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:57:47.288869 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 190/387] 10.222.0.23 () {56 vars in 1186 bytes} [Thu May 19 11:57:47 2022] GET /identity/v3/projects/0c5d68529f48460f9f6e03e3f8fb205b => generated 302 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:57:47.290718 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97708813-f0b2-499b-9e69-eacb168c878c None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:57:47.333027 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 198/388] 10.222.0.23 () {58 vars in 1023 bytes} [Thu May 19 11:57:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 11:57:47.381184 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2892ebff-6f3b-420a-a16b-0a3a4760f7d4 None None] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:57:47.413799 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:57:47.438465 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97708813-f0b2-499b-9e69-eacb168c878c None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:57:47.473694 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745b09d7-8725-4cea-9a92-ffcbf612fe21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:57:47.474949 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745b09d7-8725-4cea-9a92-ffcbf612fe21 None admin] REQUEST_METHOD: `PUT` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:47.475603 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745b09d7-8725-4cea-9a92-ffcbf612fe21 None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:47.476729 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745b09d7-8725-4cea-9a92-ffcbf612fe21 None admin] PATH_INFO: `/v3/projects/0c5d68529f48460f9f6e03e3f8fb205b/groups/5f71f3c2dd1d47f980fee625c78a2dee/roles/259eb400cdb4439db72d14456ae9cd08` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:47.479095 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745b09d7-8725-4cea-9a92-ffcbf612fe21 None admin] RBAC: Authorizing `identity:create_grant(project_id=0c5d68529f48460f9f6e03e3f8fb205b, group_id=5f71f3c2dd1d47f980fee625c78a2dee, role_id=259eb400cdb4439db72d14456ae9cd08)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:57:47.485471 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745b09d7-8725-4cea-9a92-ffcbf612fe21 None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:57:47.506981 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:57:47.555042 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2892ebff-6f3b-420a-a16b-0a3a4760f7d4 None None] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:57:47.588742 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 191/389] 10.222.0.23 () {58 vars in 1442 bytes} [Thu May 19 11:57:47 2022] PUT /identity/v3/projects/0c5d68529f48460f9f6e03e3f8fb205b/groups/5f71f3c2dd1d47f980fee625c78a2dee/roles/259eb400cdb4439db72d14456ae9cd08 => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 11:57:47.603203 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11076b6c-5474-43c9-aee1-70d029990f05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:57:47.604542 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11076b6c-5474-43c9-aee1-70d029990f05 None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:47.605016 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11076b6c-5474-43c9-aee1-70d029990f05 None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:47.605470 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11076b6c-5474-43c9-aee1-70d029990f05 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:47.619558 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11076b6c-5474-43c9-aee1-70d029990f05 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 11:57:47.629241 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application [None req-11076b6c-5474-43c9-aee1-70d029990f05 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 11:57:47.629241 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 11:57:47.629241 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 11:57:47.629241 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 11:57:47.629241 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 11:57:47.629241 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 11:57:47.629241 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 11:57:47.629241 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 11:57:47.629241 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 11:57:47.629241 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 11:57:47.629241 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 11:57:47.629241 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 11:57:47.629241 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 11:57:47.629241 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 11:57:47.629241 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 11:57:47.629241 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 11:57:47.630200 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 11:57:47.630200 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 11:57:47.630200 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 11:57:47.630200 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 11:57:47.630200 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 11:57:47.630200 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 11:57:47.630200 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 11:57:47.630200 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 11:57:47.630200 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 11:57:47.630200 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.get_or_create( May 19 11:57:47.630200 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 11:57:47.630200 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application with Lock( May 19 11:57:47.630200 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 11:57:47.630200 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._enter() May 19 11:57:47.630200 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 11:57:47.630200 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 11:57:47.631038 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 11:57:47.631038 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.creator() May 19 11:57:47.631038 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 11:57:47.631038 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application created_value = creator( May 19 11:57:47.631038 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 11:57:47.631038 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 11:57:47.631038 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 11:57:47.631038 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 11:57:47.631038 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 11:57:47.631038 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 11:57:47.631038 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 11:57:47.631038 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application May 19 11:57:47.633676 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 199/390] 10.222.0.23 () {56 vars in 1099 bytes} [Thu May 19 11:57:47 2022] GET /identity/v3/roles/member => generated 84 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 11:57:47.640946 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745b09d7-8725-4cea-9a92-ffcbf612fe21 None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:57:47.775845 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745b09d7-8725-4cea-9a92-ffcbf612fe21 None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:57:47.826198 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e6c2a0-3329-4e30-bb50-da41888d38b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:57:47.827414 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e6c2a0-3329-4e30-bb50-da41888d38b1 None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:47.828136 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e6c2a0-3329-4e30-bb50-da41888d38b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:47.828643 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e6c2a0-3329-4e30-bb50-da41888d38b1 None admin] PATH_INFO: `/v3/roles` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:47.829572 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e6c2a0-3329-4e30-bb50-da41888d38b1 None admin] RBAC: Adding query filter params (name=member) {{(pid=80539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 11:57:47.830041 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e6c2a0-3329-4e30-bb50-da41888d38b1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:57:47.831603 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e6c2a0-3329-4e30-bb50-da41888d38b1 None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:57:47.833588 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:57:47.842870 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e80ef3-622e-48bd-954d-5f848eeb8eb9 None None] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:47.843204 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e80ef3-622e-48bd-954d-5f848eeb8eb9 None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:47.843434 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e80ef3-622e-48bd-954d-5f848eeb8eb9 None None] PATH_INFO: `/` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:47.845283 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 200/391] 10.222.0.23 () {54 vars in 926 bytes} [Thu May 19 11:57:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 11:57:47.849162 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 192/392] 10.222.0.23 () {56 vars in 1101 bytes} [Thu May 19 11:57:47 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:57:47.851106 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15235678-a477-4e66-b3ba-eac52377e7f5 None None] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:47.854329 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15235678-a477-4e66-b3ba-eac52377e7f5 None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:47.854329 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15235678-a477-4e66-b3ba-eac52377e7f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:47.854785 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e6c2a0-3329-4e30-bb50-da41888d38b1 None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:57:47.889262 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.auth.core [None req-15235678-a477-4e66-b3ba-eac52377e7f5 None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 11:57:47.891374 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-15235678-a477-4e66-b3ba-eac52377e7f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 11:57:48.010230 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 201/393] 10.222.0.23 () {58 vars in 1023 bytes} [Thu May 19 11:57:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 11:57:48.016584 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e6c2a0-3329-4e30-bb50-da41888d38b1 None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:57:48.036345 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15235678-a477-4e66-b3ba-eac52377e7f5 None None] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:57:48.060569 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5743dea0-4d40-45f8-9155-f25a81a3bded None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:57:48.062224 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5743dea0-4d40-45f8-9155-f25a81a3bded None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:48.062633 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5743dea0-4d40-45f8-9155-f25a81a3bded None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:48.062912 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5743dea0-4d40-45f8-9155-f25a81a3bded None admin] PATH_INFO: `/v3/groups/5f71f3c2dd1d47f980fee625c78a2dee` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:48.064327 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5743dea0-4d40-45f8-9155-f25a81a3bded None admin] RBAC: Authorizing `identity:get_group(group_id=5f71f3c2dd1d47f980fee625c78a2dee)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:57:48.066127 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5743dea0-4d40-45f8-9155-f25a81a3bded None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:57:48.068266 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 193/394] 10.222.0.23 () {56 vars in 1180 bytes} [Thu May 19 11:57:47 2022] GET /identity/v3/groups/5f71f3c2dd1d47f980fee625c78a2dee => generated 226 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 11:57:48.075109 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5743dea0-4d40-45f8-9155-f25a81a3bded None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:57:48.114555 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:57:48.182140 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:57:48.193715 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15235678-a477-4e66-b3ba-eac52377e7f5 None None] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:57:48.225544 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5743dea0-4d40-45f8-9155-f25a81a3bded None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:57:48.234801 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c097e839-ea75-4e28-a916-f281b7302c77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:57:48.235611 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c097e839-ea75-4e28-a916-f281b7302c77 None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:48.235940 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c097e839-ea75-4e28-a916-f281b7302c77 None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:48.236223 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c097e839-ea75-4e28-a916-f281b7302c77 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:48.247998 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c097e839-ea75-4e28-a916-f281b7302c77 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 11:57:48.256876 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application [None req-c097e839-ea75-4e28-a916-f281b7302c77 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 11:57:48.256876 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 11:57:48.256876 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 11:57:48.256876 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 11:57:48.256876 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 11:57:48.256876 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 11:57:48.256876 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 11:57:48.256876 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 11:57:48.256876 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 11:57:48.256876 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 11:57:48.256876 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 11:57:48.256876 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 11:57:48.256876 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 11:57:48.256876 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 11:57:48.256876 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 11:57:48.256876 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 11:57:48.257795 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 11:57:48.257795 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 11:57:48.257795 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 11:57:48.257795 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 11:57:48.257795 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 11:57:48.257795 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 11:57:48.257795 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 11:57:48.257795 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 11:57:48.257795 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 11:57:48.257795 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.get_or_create( May 19 11:57:48.257795 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 11:57:48.257795 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application with Lock( May 19 11:57:48.257795 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 11:57:48.257795 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._enter() May 19 11:57:48.257795 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 11:57:48.257795 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 11:57:48.258567 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 11:57:48.258567 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.creator() May 19 11:57:48.258567 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 11:57:48.258567 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application created_value = creator( May 19 11:57:48.258567 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 11:57:48.258567 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 11:57:48.258567 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 11:57:48.258567 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 11:57:48.258567 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 11:57:48.258567 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 11:57:48.258567 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 11:57:48.258567 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application May 19 11:57:48.263081 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 202/395] 10.222.0.23 () {56 vars in 1114 bytes} [Thu May 19 11:57:48 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 11:57:48.264599 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5863be-6a22-4c28-8eda-43fea5610230 None None] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:48.264901 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5863be-6a22-4c28-8eda-43fea5610230 None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:48.265190 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5863be-6a22-4c28-8eda-43fea5610230 None None] PATH_INFO: `/` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:48.270426 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 203/396] 10.222.0.23 () {54 vars in 926 bytes} [Thu May 19 11:57:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 11:57:48.270426 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5863be-6a22-4c28-8eda-43fea5610230 None None] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:57:48.272576 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c5cd42-aabc-4084-a4ec-857897772348 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:57:48.273677 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c5cd42-aabc-4084-a4ec-857897772348 None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:48.274173 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c5cd42-aabc-4084-a4ec-857897772348 None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:48.274641 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c5cd42-aabc-4084-a4ec-857897772348 None admin] PATH_INFO: `/v3/projects/64092131887b4859ae47a1f4c3e0d24a` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:48.276100 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c5cd42-aabc-4084-a4ec-857897772348 None admin] RBAC: Authorizing `identity:get_project(project_id=64092131887b4859ae47a1f4c3e0d24a)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:57:48.277759 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c5cd42-aabc-4084-a4ec-857897772348 None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:57:48.279677 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 194/397] 10.222.0.23 () {56 vars in 1186 bytes} [Thu May 19 11:57:48 2022] GET /identity/v3/projects/64092131887b4859ae47a1f4c3e0d24a => generated 298 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 11:57:48.282460 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0799ae-5069-4152-84b8-65249c4f2921 None None] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:48.282963 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0799ae-5069-4152-84b8-65249c4f2921 None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:48.283430 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0799ae-5069-4152-84b8-65249c4f2921 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:48.302101 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.auth.core [None req-eb0799ae-5069-4152-84b8-65249c4f2921 None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 11:57:48.303584 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-eb0799ae-5069-4152-84b8-65249c4f2921 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 11:57:48.383907 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 195/398] 10.222.0.23 () {58 vars in 1023 bytes} [Thu May 19 11:57:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 11:57:48.385192 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0799ae-5069-4152-84b8-65249c4f2921 None None] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:57:48.387650 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5863be-6a22-4c28-8eda-43fea5610230 None None] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:57:48.392769 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:57:48.429927 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df03166-d5d9-49d3-8ed3-f405e9b8e632 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:57:48.430945 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df03166-d5d9-49d3-8ed3-f405e9b8e632 None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:48.431315 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df03166-d5d9-49d3-8ed3-f405e9b8e632 None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:48.431679 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df03166-d5d9-49d3-8ed3-f405e9b8e632 None admin] PATH_INFO: `/v3/roles` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:48.432686 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df03166-d5d9-49d3-8ed3-f405e9b8e632 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 11:57:48.433069 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df03166-d5d9-49d3-8ed3-f405e9b8e632 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:57:48.434776 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df03166-d5d9-49d3-8ed3-f405e9b8e632 None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:57:48.445373 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 204/399] 10.222.0.23 () {56 vars in 1111 bytes} [Thu May 19 11:57:48 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:57:48.446151 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df03166-d5d9-49d3-8ed3-f405e9b8e632 None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:57:48.470391 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:57:48.513876 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0799ae-5069-4152-84b8-65249c4f2921 None None] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:57:48.550762 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350de265-d9b1-42e3-aac8-e26c781bdae3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:57:48.552228 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350de265-d9b1-42e3-aac8-e26c781bdae3 None admin] REQUEST_METHOD: `PUT` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:48.552681 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350de265-d9b1-42e3-aac8-e26c781bdae3 None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:48.553127 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350de265-d9b1-42e3-aac8-e26c781bdae3 None admin] PATH_INFO: `/v3/projects/64092131887b4859ae47a1f4c3e0d24a/groups/5f71f3c2dd1d47f980fee625c78a2dee/roles/259eb400cdb4439db72d14456ae9cd08` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:48.555275 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-350de265-d9b1-42e3-aac8-e26c781bdae3 None admin] RBAC: Authorizing `identity:create_grant(project_id=64092131887b4859ae47a1f4c3e0d24a, group_id=5f71f3c2dd1d47f980fee625c78a2dee, role_id=259eb400cdb4439db72d14456ae9cd08)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:57:48.556936 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-350de265-d9b1-42e3-aac8-e26c781bdae3 None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:57:48.562607 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df03166-d5d9-49d3-8ed3-f405e9b8e632 None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:57:48.598703 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361af315-0343-4688-9852-f7dbefb3b3ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:57:48.599672 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361af315-0343-4688-9852-f7dbefb3b3ea None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:48.600059 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361af315-0343-4688-9852-f7dbefb3b3ea None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:48.600456 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361af315-0343-4688-9852-f7dbefb3b3ea None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:48.614943 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361af315-0343-4688-9852-f7dbefb3b3ea None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 11:57:48.623604 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application [None req-361af315-0343-4688-9852-f7dbefb3b3ea None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 11:57:48.623604 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 11:57:48.623604 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 11:57:48.623604 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 11:57:48.623604 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 11:57:48.623604 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 11:57:48.623604 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 11:57:48.623604 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 11:57:48.623604 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 11:57:48.623604 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 11:57:48.623604 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 11:57:48.623604 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 11:57:48.623604 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 11:57:48.623604 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 11:57:48.623604 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 11:57:48.623604 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 11:57:48.625402 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 11:57:48.625402 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 11:57:48.625402 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 11:57:48.625402 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 11:57:48.625402 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 11:57:48.625402 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 11:57:48.625402 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 11:57:48.625402 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 11:57:48.625402 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 11:57:48.625402 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.get_or_create( May 19 11:57:48.625402 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 11:57:48.625402 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application with Lock( May 19 11:57:48.625402 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 11:57:48.625402 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._enter() May 19 11:57:48.625402 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 11:57:48.625402 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 11:57:48.626193 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 11:57:48.626193 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.creator() May 19 11:57:48.626193 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 11:57:48.626193 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application created_value = creator( May 19 11:57:48.626193 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 11:57:48.626193 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 11:57:48.626193 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 11:57:48.626193 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 11:57:48.626193 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 11:57:48.626193 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 11:57:48.626193 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 11:57:48.626193 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application May 19 11:57:48.626193 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 205/400] 10.222.0.23 () {56 vars in 1114 bytes} [Thu May 19 11:57:48 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 11:57:48.632217 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361af315-0343-4688-9852-f7dbefb3b3ea None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:57:48.640116 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 196/401] 10.222.0.23 () {58 vars in 1442 bytes} [Thu May 19 11:57:48 2022] PUT /identity/v3/projects/64092131887b4859ae47a1f4c3e0d24a/groups/5f71f3c2dd1d47f980fee625c78a2dee/roles/259eb400cdb4439db72d14456ae9cd08 => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 11:57:48.640672 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350de265-d9b1-42e3-aac8-e26c781bdae3 None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:57:48.685548 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:57:48.779284 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350de265-d9b1-42e3-aac8-e26c781bdae3 None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:57:48.812972 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:57:48.830819 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd547308-152e-4133-95e0-1a9ff8719ab3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:57:48.832182 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd547308-152e-4133-95e0-1a9ff8719ab3 None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:48.832695 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd547308-152e-4133-95e0-1a9ff8719ab3 None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:48.833638 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd547308-152e-4133-95e0-1a9ff8719ab3 None admin] PATH_INFO: `/v3/roles` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:48.834979 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd547308-152e-4133-95e0-1a9ff8719ab3 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 11:57:48.835818 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd547308-152e-4133-95e0-1a9ff8719ab3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:57:48.838280 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd547308-152e-4133-95e0-1a9ff8719ab3 None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:57:48.844787 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361af315-0343-4688-9852-f7dbefb3b3ea None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:57:48.852076 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 197/402] 10.222.0.23 () {56 vars in 1111 bytes} [Thu May 19 11:57:48 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:57:48.853923 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3593ce-f05e-4fd2-b8e6-3f2ec880e7f1 None None] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:48.855074 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3593ce-f05e-4fd2-b8e6-3f2ec880e7f1 None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:48.855597 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3593ce-f05e-4fd2-b8e6-3f2ec880e7f1 None None] PATH_INFO: `/` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:48.857383 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 198/403] 10.222.0.23 () {54 vars in 926 bytes} [Thu May 19 11:57:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 11:57:48.868053 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3593ce-f05e-4fd2-b8e6-3f2ec880e7f1 None None] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:57:48.893323 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d24f9e4-1347-4f89-a392-34cb4adf55ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:57:48.894465 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d24f9e4-1347-4f89-a392-34cb4adf55ae None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:48.894960 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d24f9e4-1347-4f89-a392-34cb4adf55ae None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:48.895321 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d24f9e4-1347-4f89-a392-34cb4adf55ae None admin] PATH_INFO: `/v3/groups/5f71f3c2dd1d47f980fee625c78a2dee` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:48.896777 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d24f9e4-1347-4f89-a392-34cb4adf55ae None admin] RBAC: Authorizing `identity:get_group(group_id=5f71f3c2dd1d47f980fee625c78a2dee)` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:57:48.898859 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d24f9e4-1347-4f89-a392-34cb4adf55ae None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:57:48.900091 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 206/404] 10.222.0.23 () {56 vars in 1180 bytes} [Thu May 19 11:57:48 2022] GET /identity/v3/groups/5f71f3c2dd1d47f980fee625c78a2dee => generated 226 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:57:48.906551 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309f6e39-db7e-4074-877e-3ce28785802b None None] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:48.906928 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309f6e39-db7e-4074-877e-3ce28785802b None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:48.907286 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309f6e39-db7e-4074-877e-3ce28785802b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:48.925465 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.auth.core [None req-309f6e39-db7e-4074-877e-3ce28785802b None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 11:57:48.928724 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-309f6e39-db7e-4074-877e-3ce28785802b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 11:57:48.984324 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3593ce-f05e-4fd2-b8e6-3f2ec880e7f1 None None] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:57:49.024542 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 207/405] 10.222.0.23 () {58 vars in 1023 bytes} [Thu May 19 11:57:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 11:57:49.025695 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309f6e39-db7e-4074-877e-3ce28785802b None None] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:57:49.032667 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb2bd4f-63cd-4d8a-b1a8-b33c46e1bb16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:57:49.033641 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb2bd4f-63cd-4d8a-b1a8-b33c46e1bb16 None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:49.033992 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb2bd4f-63cd-4d8a-b1a8-b33c46e1bb16 None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:49.034358 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb2bd4f-63cd-4d8a-b1a8-b33c46e1bb16 None admin] PATH_INFO: `/v3/groups/5f71f3c2dd1d47f980fee625c78a2dee` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:49.037976 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb2bd4f-63cd-4d8a-b1a8-b33c46e1bb16 None admin] RBAC: Authorizing `identity:get_group(group_id=5f71f3c2dd1d47f980fee625c78a2dee)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:57:49.039513 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb2bd4f-63cd-4d8a-b1a8-b33c46e1bb16 None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:57:49.041763 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 199/406] 10.222.0.23 () {56 vars in 1180 bytes} [Thu May 19 11:57:48 2022] GET /identity/v3/groups/5f71f3c2dd1d47f980fee625c78a2dee => generated 226 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:57:49.047303 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb2bd4f-63cd-4d8a-b1a8-b33c46e1bb16 None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:57:49.060471 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:57:49.120578 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:57:49.146444 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309f6e39-db7e-4074-877e-3ce28785802b None None] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:57:49.173390 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb2bd4f-63cd-4d8a-b1a8-b33c46e1bb16 None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:57:49.188438 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9449e092-3eb1-4031-b430-210c5b618112 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:57:49.189320 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9449e092-3eb1-4031-b430-210c5b618112 None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:49.189609 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9449e092-3eb1-4031-b430-210c5b618112 None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:49.189892 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9449e092-3eb1-4031-b430-210c5b618112 None admin] PATH_INFO: `/v3/projects/64092131887b4859ae47a1f4c3e0d24a` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:49.191232 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9449e092-3eb1-4031-b430-210c5b618112 None admin] RBAC: Authorizing `identity:get_project(project_id=64092131887b4859ae47a1f4c3e0d24a)` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:57:49.192946 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9449e092-3eb1-4031-b430-210c5b618112 None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:57:49.195297 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 208/407] 10.222.0.23 () {56 vars in 1186 bytes} [Thu May 19 11:57:49 2022] GET /identity/v3/projects/64092131887b4859ae47a1f4c3e0d24a => generated 298 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:57:49.196118 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9449e092-3eb1-4031-b430-210c5b618112 None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:57:49.211142 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b3b0d3-6508-4c47-bb4d-7c07eff85e9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:57:49.212089 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b3b0d3-6508-4c47-bb4d-7c07eff85e9e None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:49.212374 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b3b0d3-6508-4c47-bb4d-7c07eff85e9e None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:49.212673 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b3b0d3-6508-4c47-bb4d-7c07eff85e9e None admin] PATH_INFO: `/v3/projects/0c5d68529f48460f9f6e03e3f8fb205b` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:49.214048 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b3b0d3-6508-4c47-bb4d-7c07eff85e9e None admin] RBAC: Authorizing `identity:get_project(project_id=0c5d68529f48460f9f6e03e3f8fb205b)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:57:49.216552 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b3b0d3-6508-4c47-bb4d-7c07eff85e9e None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:57:49.219736 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 200/408] 10.222.0.23 () {56 vars in 1186 bytes} [Thu May 19 11:57:49 2022] GET /identity/v3/projects/0c5d68529f48460f9f6e03e3f8fb205b => generated 302 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 11:57:49.220662 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b3b0d3-6508-4c47-bb4d-7c07eff85e9e None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:57:49.298922 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:57:49.308007 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9449e092-3eb1-4031-b430-210c5b618112 None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:57:49.338249 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b3b0d3-6508-4c47-bb4d-7c07eff85e9e None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:57:49.346660 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24f591f-b6e4-4e10-8266-55a43d95e375 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:57:49.347716 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24f591f-b6e4-4e10-8266-55a43d95e375 None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:49.347868 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24f591f-b6e4-4e10-8266-55a43d95e375 None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:49.348155 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24f591f-b6e4-4e10-8266-55a43d95e375 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:49.360357 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24f591f-b6e4-4e10-8266-55a43d95e375 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 11:57:49.362817 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:57:49.369179 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application [None req-d24f591f-b6e4-4e10-8266-55a43d95e375 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 11:57:49.369179 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 11:57:49.369179 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 11:57:49.369179 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 11:57:49.369179 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 11:57:49.369179 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 11:57:49.369179 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 11:57:49.369179 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 11:57:49.369179 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 11:57:49.369179 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 11:57:49.369179 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 11:57:49.369179 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 11:57:49.369179 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 11:57:49.369179 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 11:57:49.369179 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 11:57:49.369179 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 11:57:49.370231 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 11:57:49.370231 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 11:57:49.370231 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 11:57:49.370231 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 11:57:49.370231 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 11:57:49.370231 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 11:57:49.370231 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 11:57:49.370231 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 11:57:49.370231 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 11:57:49.370231 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.get_or_create( May 19 11:57:49.370231 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 11:57:49.370231 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application with Lock( May 19 11:57:49.370231 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 11:57:49.370231 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._enter() May 19 11:57:49.370231 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 11:57:49.370231 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 11:57:49.371217 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 11:57:49.371217 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.creator() May 19 11:57:49.371217 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 11:57:49.371217 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application created_value = creator( May 19 11:57:49.371217 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 11:57:49.371217 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 11:57:49.371217 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 11:57:49.371217 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 11:57:49.371217 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 11:57:49.371217 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 11:57:49.371217 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 11:57:49.371217 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application May 19 11:57:49.375826 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 209/409] 10.222.0.23 () {56 vars in 1096 bytes} [Thu May 19 11:57:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 11:57:49.376747 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24f591f-b6e4-4e10-8266-55a43d95e375 None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:57:49.380903 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95508e6d-0ffb-4a2f-8c3a-7869e5156854 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:57:49.381992 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95508e6d-0ffb-4a2f-8c3a-7869e5156854 None admin] REQUEST_METHOD: `PUT` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:49.382421 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95508e6d-0ffb-4a2f-8c3a-7869e5156854 None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:49.382840 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95508e6d-0ffb-4a2f-8c3a-7869e5156854 None admin] PATH_INFO: `/v3/projects/64092131887b4859ae47a1f4c3e0d24a/groups/5f71f3c2dd1d47f980fee625c78a2dee/roles/8215583bb4224c75b093ae8e3993eb9f` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:49.385329 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95508e6d-0ffb-4a2f-8c3a-7869e5156854 None admin] RBAC: Authorizing `identity:create_grant(project_id=64092131887b4859ae47a1f4c3e0d24a, group_id=5f71f3c2dd1d47f980fee625c78a2dee, role_id=8215583bb4224c75b093ae8e3993eb9f)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:57:49.387048 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95508e6d-0ffb-4a2f-8c3a-7869e5156854 None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:57:49.471619 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 201/410] 10.222.0.23 () {58 vars in 1442 bytes} [Thu May 19 11:57:49 2022] PUT /identity/v3/projects/64092131887b4859ae47a1f4c3e0d24a/groups/5f71f3c2dd1d47f980fee625c78a2dee/roles/8215583bb4224c75b093ae8e3993eb9f => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 11:57:49.472919 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95508e6d-0ffb-4a2f-8c3a-7869e5156854 None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:57:49.490264 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24f591f-b6e4-4e10-8266-55a43d95e375 None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:57:49.571373 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403787bf-d67c-4b9a-bf64-ec211c9830c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:57:49.574224 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403787bf-d67c-4b9a-bf64-ec211c9830c5 None admin] REQUEST_METHOD: `PUT` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:49.575016 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403787bf-d67c-4b9a-bf64-ec211c9830c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:49.576045 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403787bf-d67c-4b9a-bf64-ec211c9830c5 None admin] PATH_INFO: `/v3/projects/0c5d68529f48460f9f6e03e3f8fb205b/groups/5f71f3c2dd1d47f980fee625c78a2dee/roles/8215583bb4224c75b093ae8e3993eb9f` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:49.583296 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403787bf-d67c-4b9a-bf64-ec211c9830c5 None admin] RBAC: Authorizing `identity:create_grant(project_id=0c5d68529f48460f9f6e03e3f8fb205b, group_id=5f71f3c2dd1d47f980fee625c78a2dee, role_id=8215583bb4224c75b093ae8e3993eb9f)` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:57:49.585595 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403787bf-d67c-4b9a-bf64-ec211c9830c5 None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:57:49.647727 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95508e6d-0ffb-4a2f-8c3a-7869e5156854 None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:57:49.658755 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 210/411] 10.222.0.23 () {58 vars in 1442 bytes} [Thu May 19 11:57:49 2022] PUT /identity/v3/projects/0c5d68529f48460f9f6e03e3f8fb205b/groups/5f71f3c2dd1d47f980fee625c78a2dee/roles/8215583bb4224c75b093ae8e3993eb9f => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 11:57:49.706102 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0f0652-81f9-4f15-96a7-e4b68bb15d8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:57:49.707167 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0f0652-81f9-4f15-96a7-e4b68bb15d8f None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:49.707766 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0f0652-81f9-4f15-96a7-e4b68bb15d8f None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:49.708349 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0f0652-81f9-4f15-96a7-e4b68bb15d8f None admin] PATH_INFO: `/v3/roles` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:49.716430 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0f0652-81f9-4f15-96a7-e4b68bb15d8f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 11:57:49.716430 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0f0652-81f9-4f15-96a7-e4b68bb15d8f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:57:49.716430 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0f0652-81f9-4f15-96a7-e4b68bb15d8f None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:57:49.723414 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 202/412] 10.222.0.23 () {56 vars in 1099 bytes} [Thu May 19 11:57:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:57:49.730602 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403787bf-d67c-4b9a-bf64-ec211c9830c5 None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:57:49.754415 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:57:49.860346 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403787bf-d67c-4b9a-bf64-ec211c9830c5 None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:57:49.901313 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4941c9-22d9-4f10-afc0-db77e12854ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:57:49.902707 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4941c9-22d9-4f10-afc0-db77e12854ad None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:49.903206 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4941c9-22d9-4f10-afc0-db77e12854ad None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:49.903668 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4941c9-22d9-4f10-afc0-db77e12854ad None admin] PATH_INFO: `/v3/groups/2b83c34e4dc54b2fb1afe8a56166cd2c` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:49.905345 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4941c9-22d9-4f10-afc0-db77e12854ad None admin] RBAC: Authorizing `identity:get_group(group_id=2b83c34e4dc54b2fb1afe8a56166cd2c)` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:57:49.907011 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4941c9-22d9-4f10-afc0-db77e12854ad None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:57:49.909386 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 211/413] 10.222.0.23 () {56 vars in 1180 bytes} [Thu May 19 11:57:49 2022] GET /identity/v3/groups/2b83c34e4dc54b2fb1afe8a56166cd2c => generated 229 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:57:49.914626 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0f0652-81f9-4f15-96a7-e4b68bb15d8f None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:57:49.964780 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:57:50.031300 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0f0652-81f9-4f15-96a7-e4b68bb15d8f None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:57:50.068754 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f60a99-748e-4acb-9e2c-1692d21554b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:57:50.069629 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f60a99-748e-4acb-9e2c-1692d21554b9 None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:50.070166 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f60a99-748e-4acb-9e2c-1692d21554b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:50.070446 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f60a99-748e-4acb-9e2c-1692d21554b9 None admin] PATH_INFO: `/v3/projects/f3569780f2fe4b24b4109e2eee3c26fe` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:50.071723 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f60a99-748e-4acb-9e2c-1692d21554b9 None admin] RBAC: Authorizing `identity:get_project(project_id=f3569780f2fe4b24b4109e2eee3c26fe)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:57:50.073323 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f60a99-748e-4acb-9e2c-1692d21554b9 None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:57:50.075409 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 203/414] 10.222.0.23 () {56 vars in 1186 bytes} [Thu May 19 11:57:49 2022] GET /identity/v3/projects/f3569780f2fe4b24b4109e2eee3c26fe => generated 344 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:57:50.081072 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4941c9-22d9-4f10-afc0-db77e12854ad None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:57:50.205732 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:57:50.228205 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4941c9-22d9-4f10-afc0-db77e12854ad None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:57:50.265727 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b8c2bc-473a-4c73-8802-bb09ea760557 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:57:50.266356 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b8c2bc-473a-4c73-8802-bb09ea760557 None admin] REQUEST_METHOD: `PUT` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:50.266647 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b8c2bc-473a-4c73-8802-bb09ea760557 None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:50.266924 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b8c2bc-473a-4c73-8802-bb09ea760557 None admin] PATH_INFO: `/v3/projects/f3569780f2fe4b24b4109e2eee3c26fe/groups/2b83c34e4dc54b2fb1afe8a56166cd2c/roles/9db5671c65d3414ea2682afa2ccf1403` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:50.269127 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b8c2bc-473a-4c73-8802-bb09ea760557 None admin] RBAC: Authorizing `identity:create_grant(project_id=f3569780f2fe4b24b4109e2eee3c26fe, group_id=2b83c34e4dc54b2fb1afe8a56166cd2c, role_id=9db5671c65d3414ea2682afa2ccf1403)` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:57:50.270652 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b8c2bc-473a-4c73-8802-bb09ea760557 None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:57:50.285614 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8ee9ee-1a56-439f-b3b3-f796838f17ec None None] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:50.285905 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8ee9ee-1a56-439f-b3b3-f796838f17ec None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:50.288673 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8ee9ee-1a56-439f-b3b3-f796838f17ec None None] PATH_INFO: `/` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:50.288673 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 204/415] 10.222.0.23 () {54 vars in 926 bytes} [Thu May 19 11:57:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 11:57:50.293026 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1930e8c4-8d8b-43e6-a54c-6a9c4fae2da9 None None] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:50.293314 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1930e8c4-8d8b-43e6-a54c-6a9c4fae2da9 None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:50.293592 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1930e8c4-8d8b-43e6-a54c-6a9c4fae2da9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:50.312299 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.auth.core [None req-1930e8c4-8d8b-43e6-a54c-6a9c4fae2da9 None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 11:57:50.328007 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-1930e8c4-8d8b-43e6-a54c-6a9c4fae2da9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 11:57:50.355023 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 212/416] 10.222.0.23 () {58 vars in 1442 bytes} [Thu May 19 11:57:50 2022] PUT /identity/v3/projects/f3569780f2fe4b24b4109e2eee3c26fe/groups/2b83c34e4dc54b2fb1afe8a56166cd2c/roles/9db5671c65d3414ea2682afa2ccf1403 => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 11:57:50.410830 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 205/417] 10.222.0.23 () {58 vars in 1023 bytes} [Thu May 19 11:57:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 11:57:50.456144 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b8c2bc-473a-4c73-8802-bb09ea760557 None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:57:50.601811 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b8c2bc-473a-4c73-8802-bb09ea760557 None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:57:50.642813 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd6389a-3156-4234-b199-22f1356798ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:57:50.643635 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd6389a-3156-4234-b199-22f1356798ec None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:50.643957 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd6389a-3156-4234-b199-22f1356798ec None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:50.644235 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd6389a-3156-4234-b199-22f1356798ec None admin] PATH_INFO: `/v3/roles/member` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:50.652952 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:57:50.657505 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd6389a-3156-4234-b199-22f1356798ec None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 11:57:50.665619 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application [None req-2bd6389a-3156-4234-b199-22f1356798ec None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 11:57:50.665619 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 11:57:50.665619 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 11:57:50.665619 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 11:57:50.665619 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 11:57:50.665619 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 11:57:50.665619 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 11:57:50.665619 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 11:57:50.665619 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 11:57:50.665619 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 11:57:50.665619 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 11:57:50.665619 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 11:57:50.665619 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 11:57:50.665619 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 11:57:50.665619 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 11:57:50.665619 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 11:57:50.667034 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 11:57:50.667034 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 11:57:50.667034 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 11:57:50.667034 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 11:57:50.667034 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 11:57:50.667034 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 11:57:50.667034 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 11:57:50.667034 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 11:57:50.667034 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 11:57:50.667034 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.get_or_create( May 19 11:57:50.667034 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 11:57:50.667034 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application with Lock( May 19 11:57:50.667034 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 11:57:50.667034 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._enter() May 19 11:57:50.667034 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 11:57:50.667034 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 11:57:50.667849 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 11:57:50.667849 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.creator() May 19 11:57:50.667849 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 11:57:50.667849 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application created_value = creator( May 19 11:57:50.667849 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 11:57:50.667849 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 11:57:50.667849 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 11:57:50.667849 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 11:57:50.667849 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 11:57:50.667849 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 11:57:50.667849 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 11:57:50.667849 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application May 19 11:57:50.667849 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 213/418] 10.222.0.23 () {56 vars in 1099 bytes} [Thu May 19 11:57:50 2022] GET /identity/v3/roles/member => generated 84 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 11:57:50.673985 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1930e8c4-8d8b-43e6-a54c-6a9c4fae2da9 None None] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:57:50.723564 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:57:50.800422 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1930e8c4-8d8b-43e6-a54c-6a9c4fae2da9 None None] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:57:50.848268 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f11d27-b5b3-4220-ba3a-fc4a1348bae2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:57:50.848941 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f11d27-b5b3-4220-ba3a-fc4a1348bae2 None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:50.849222 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f11d27-b5b3-4220-ba3a-fc4a1348bae2 None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:50.849500 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f11d27-b5b3-4220-ba3a-fc4a1348bae2 None admin] PATH_INFO: `/v3/roles` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:50.850251 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f11d27-b5b3-4220-ba3a-fc4a1348bae2 None admin] RBAC: Adding query filter params (name=member) {{(pid=80539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 11:57:50.850555 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f11d27-b5b3-4220-ba3a-fc4a1348bae2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:57:50.852096 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f11d27-b5b3-4220-ba3a-fc4a1348bae2 None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:57:50.864590 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 206/419] 10.222.0.23 () {56 vars in 1101 bytes} [Thu May 19 11:57:50 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:57:50.869999 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd6389a-3156-4234-b199-22f1356798ec None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:57:51.028838 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd6389a-3156-4234-b199-22f1356798ec None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:57:51.066259 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f521838c-7fe9-46d4-9d37-34cbfadb9f7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:57:51.066927 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f521838c-7fe9-46d4-9d37-34cbfadb9f7d None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:51.067219 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f521838c-7fe9-46d4-9d37-34cbfadb9f7d None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:51.067503 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f521838c-7fe9-46d4-9d37-34cbfadb9f7d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:51.173266 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:57:51.181255 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f521838c-7fe9-46d4-9d37-34cbfadb9f7d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:57:51.182804 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f521838c-7fe9-46d4-9d37-34cbfadb9f7d None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:57:51.334083 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 214/420] 10.222.0.23 () {58 vars in 1282 bytes} [Thu May 19 11:57:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 464 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 19 11:57:51.339431 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f11d27-b5b3-4220-ba3a-fc4a1348bae2 None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:57:51.420314 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cd3791-ec32-43bc-8bdd-795871e5fb3d None None] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:51.421284 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cd3791-ec32-43bc-8bdd-795871e5fb3d None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:51.421573 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cd3791-ec32-43bc-8bdd-795871e5fb3d None None] PATH_INFO: `/` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:51.423889 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 215/421] 10.222.0.23 () {54 vars in 926 bytes} [Thu May 19 11:57:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 11:57:51.428782 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05817492-7d13-428c-af0d-cce1e0610f0c None None] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:51.429263 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05817492-7d13-428c-af0d-cce1e0610f0c None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:51.429555 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05817492-7d13-428c-af0d-cce1e0610f0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:51.454537 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.auth.core [None req-05817492-7d13-428c-af0d-cce1e0610f0c None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 11:57:51.455750 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-05817492-7d13-428c-af0d-cce1e0610f0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 11:57:51.540352 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f11d27-b5b3-4220-ba3a-fc4a1348bae2 None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:57:51.542875 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:57:51.593202 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 216/422] 10.222.0.23 () {58 vars in 1023 bytes} [Thu May 19 11:57:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 11:57:51.601386 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a31cb2-9175-48e3-aed0-6bf88112a934 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:57:51.602510 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a31cb2-9175-48e3-aed0-6bf88112a934 None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:51.602990 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a31cb2-9175-48e3-aed0-6bf88112a934 None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:51.603480 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a31cb2-9175-48e3-aed0-6bf88112a934 None admin] PATH_INFO: `/v3/projects/0c5d68529f48460f9f6e03e3f8fb205b` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:51.604904 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a31cb2-9175-48e3-aed0-6bf88112a934 None admin] RBAC: Authorizing `identity:get_project(project_id=0c5d68529f48460f9f6e03e3f8fb205b)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:57:51.606549 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a31cb2-9175-48e3-aed0-6bf88112a934 None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:57:51.609478 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 207/423] 10.222.0.23 () {56 vars in 1186 bytes} [Thu May 19 11:57:51 2022] GET /identity/v3/projects/0c5d68529f48460f9f6e03e3f8fb205b => generated 302 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:57:51.617090 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05817492-7d13-428c-af0d-cce1e0610f0c None None] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:57:51.636670 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a31cb2-9175-48e3-aed0-6bf88112a934 None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:57:51.652813 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:57:51.727373 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05817492-7d13-428c-af0d-cce1e0610f0c None None] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:57:51.771827 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2122ff0d-c26a-4292-aacb-05edf5acfd9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:57:51.771827 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2122ff0d-c26a-4292-aacb-05edf5acfd9c None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:51.771827 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2122ff0d-c26a-4292-aacb-05edf5acfd9c None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:51.771827 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2122ff0d-c26a-4292-aacb-05edf5acfd9c None admin] PATH_INFO: `/v3/groups/5f71f3c2dd1d47f980fee625c78a2dee` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:51.772278 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2122ff0d-c26a-4292-aacb-05edf5acfd9c None admin] RBAC: Authorizing `identity:get_group(group_id=5f71f3c2dd1d47f980fee625c78a2dee)` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:57:51.772278 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2122ff0d-c26a-4292-aacb-05edf5acfd9c None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:57:51.772278 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 217/424] 10.222.0.23 () {56 vars in 1180 bytes} [Thu May 19 11:57:51 2022] GET /identity/v3/groups/5f71f3c2dd1d47f980fee625c78a2dee => generated 226 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:57:51.777627 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2122ff0d-c26a-4292-aacb-05edf5acfd9c None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:57:51.818311 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a31cb2-9175-48e3-aed0-6bf88112a934 None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:57:51.908513 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560571f7-fa11-4c56-a4ac-12561e015ef7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:57:51.909597 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560571f7-fa11-4c56-a4ac-12561e015ef7 None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:51.910039 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560571f7-fa11-4c56-a4ac-12561e015ef7 None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:51.910488 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560571f7-fa11-4c56-a4ac-12561e015ef7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:51.924353 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560571f7-fa11-4c56-a4ac-12561e015ef7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 11:57:51.934039 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application [None req-560571f7-fa11-4c56-a4ac-12561e015ef7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 11:57:51.934039 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 11:57:51.934039 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 11:57:51.934039 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 11:57:51.934039 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 11:57:51.934039 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 11:57:51.934039 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 11:57:51.934039 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 11:57:51.934039 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 11:57:51.934039 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 11:57:51.934039 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 11:57:51.934039 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 11:57:51.934039 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 11:57:51.934039 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 11:57:51.934039 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 11:57:51.934039 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 11:57:51.935002 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 11:57:51.935002 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 11:57:51.935002 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 11:57:51.935002 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 11:57:51.935002 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 11:57:51.935002 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 11:57:51.935002 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 11:57:51.935002 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 11:57:51.935002 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 11:57:51.935002 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.get_or_create( May 19 11:57:51.935002 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 11:57:51.935002 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application with Lock( May 19 11:57:51.935002 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 11:57:51.935002 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._enter() May 19 11:57:51.935002 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 11:57:51.935002 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 11:57:51.935844 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 11:57:51.935844 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.creator() May 19 11:57:51.935844 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 11:57:51.935844 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application created_value = creator( May 19 11:57:51.935844 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 11:57:51.935844 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 11:57:51.935844 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 11:57:51.935844 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 11:57:51.935844 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 11:57:51.935844 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 11:57:51.935844 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 11:57:51.935844 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application May 19 11:57:51.939479 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 208/425] 10.222.0.23 () {56 vars in 1099 bytes} [Thu May 19 11:57:51 2022] GET /identity/v3/roles/member => generated 84 bytes in 303 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 11:57:51.945402 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560571f7-fa11-4c56-a4ac-12561e015ef7 None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:57:51.976780 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:57:51.983549 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2122ff0d-c26a-4292-aacb-05edf5acfd9c None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:57:52.027587 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:57:52.032273 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0239b52f-5af6-4077-9fa4-b74c17541e00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:57:52.033102 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0239b52f-5af6-4077-9fa4-b74c17541e00 None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:52.033997 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0239b52f-5af6-4077-9fa4-b74c17541e00 None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:52.034493 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0239b52f-5af6-4077-9fa4-b74c17541e00 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:52.037457 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0239b52f-5af6-4077-9fa4-b74c17541e00 None admin] RBAC: Adding query filter params (group.id=5f71f3c2dd1d47f980fee625c78a2dee, role.id=259eb400cdb4439db72d14456ae9cd08, scope.project.id=0c5d68529f48460f9f6e03e3f8fb205b) {{(pid=80540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 11:57:52.038969 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0239b52f-5af6-4077-9fa4-b74c17541e00 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:57:52.039640 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0239b52f-5af6-4077-9fa4-b74c17541e00 None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:57:52.048716 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 218/426] 10.222.0.23 () {56 vars in 1376 bytes} [Thu May 19 11:57:51 2022] GET /identity/v3/role_assignments?group.id=5f71f3c2dd1d47f980fee625c78a2dee&scope.project.id=0c5d68529f48460f9f6e03e3f8fb205b&role.id=259eb400cdb4439db72d14456ae9cd08 => generated 611 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:57:52.145666 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560571f7-fa11-4c56-a4ac-12561e015ef7 None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:57:52.207708 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5f6c20-2dd5-40d0-b066-2c9548364a86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:57:52.208351 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5f6c20-2dd5-40d0-b066-2c9548364a86 None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:52.208795 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5f6c20-2dd5-40d0-b066-2c9548364a86 None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:52.208960 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5f6c20-2dd5-40d0-b066-2c9548364a86 None admin] PATH_INFO: `/v3/roles` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:52.209695 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5f6c20-2dd5-40d0-b066-2c9548364a86 None admin] RBAC: Adding query filter params (name=member) {{(pid=80539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 11:57:52.210009 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5f6c20-2dd5-40d0-b066-2c9548364a86 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:57:52.211461 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5f6c20-2dd5-40d0-b066-2c9548364a86 None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:57:52.229056 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0239b52f-5af6-4077-9fa4-b74c17541e00 None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:57:52.229517 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 209/427] 10.222.0.23 () {56 vars in 1101 bytes} [Thu May 19 11:57:51 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:57:52.371252 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0239b52f-5af6-4077-9fa4-b74c17541e00 None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:57:52.412668 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55bf2b3b-425c-49f5-8bfc-21f540187112 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:57:52.413729 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bf2b3b-425c-49f5-8bfc-21f540187112 None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:52.414214 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bf2b3b-425c-49f5-8bfc-21f540187112 None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:52.414686 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bf2b3b-425c-49f5-8bfc-21f540187112 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:52.451397 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:57:52.504931 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55bf2b3b-425c-49f5-8bfc-21f540187112 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:57:52.506797 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55bf2b3b-425c-49f5-8bfc-21f540187112 None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:57:52.615248 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a995f63-a52b-4a85-b7a3-1c92382abf90 None None] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:52.615248 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a995f63-a52b-4a85-b7a3-1c92382abf90 None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:52.615248 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a995f63-a52b-4a85-b7a3-1c92382abf90 None None] PATH_INFO: `/` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:52.616447 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 210/428] 10.222.0.23 () {54 vars in 926 bytes} [Thu May 19 11:57:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 11:57:52.624265 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7b1825-23c5-4ef3-a6ae-6ed5017b82f0 None None] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:52.624265 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7b1825-23c5-4ef3-a6ae-6ed5017b82f0 None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:52.624537 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7b1825-23c5-4ef3-a6ae-6ed5017b82f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:52.642767 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.auth.core [None req-5a7b1825-23c5-4ef3-a6ae-6ed5017b82f0 None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 11:57:52.644044 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-5a7b1825-23c5-4ef3-a6ae-6ed5017b82f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 11:57:52.648658 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 219/429] 10.222.0.23 () {58 vars in 1282 bytes} [Thu May 19 11:57:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 420 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 11:57:52.657312 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55bf2b3b-425c-49f5-8bfc-21f540187112 None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:57:52.731352 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 211/430] 10.222.0.23 () {58 vars in 1023 bytes} [Thu May 19 11:57:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 11:57:52.733439 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9f2b1f-8b1a-48a5-91a1-33e6ec9d381a None None] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:52.733927 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9f2b1f-8b1a-48a5-91a1-33e6ec9d381a None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:52.734383 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9f2b1f-8b1a-48a5-91a1-33e6ec9d381a None None] PATH_INFO: `/` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:52.736284 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 212/431] 10.222.0.23 () {54 vars in 926 bytes} [Thu May 19 11:57:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 11:57:52.742301 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e8732f-6432-4dc0-a3e1-f1a81cbb2405 None None] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:52.742856 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e8732f-6432-4dc0-a3e1-f1a81cbb2405 None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:52.743309 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e8732f-6432-4dc0-a3e1-f1a81cbb2405 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:52.746798 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:57:52.762538 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.auth.core [None req-34e8732f-6432-4dc0-a3e1-f1a81cbb2405 None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 11:57:52.763661 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-34e8732f-6432-4dc0-a3e1-f1a81cbb2405 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 11:57:52.778174 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55bf2b3b-425c-49f5-8bfc-21f540187112 None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:57:52.785680 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:57:52.816909 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b2fa7e-84a7-4a4a-a8b2-a4032bd16fe6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:57:52.817790 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b2fa7e-84a7-4a4a-a8b2-a4032bd16fe6 None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:52.818075 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b2fa7e-84a7-4a4a-a8b2-a4032bd16fe6 None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:52.818360 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b2fa7e-84a7-4a4a-a8b2-a4032bd16fe6 None admin] PATH_INFO: `/v3/projects/64092131887b4859ae47a1f4c3e0d24a` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:52.819584 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b2fa7e-84a7-4a4a-a8b2-a4032bd16fe6 None admin] RBAC: Authorizing `identity:get_project(project_id=64092131887b4859ae47a1f4c3e0d24a)` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:57:52.821405 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b2fa7e-84a7-4a4a-a8b2-a4032bd16fe6 None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:57:52.828379 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 220/432] 10.222.0.23 () {56 vars in 1186 bytes} [Thu May 19 11:57:52 2022] GET /identity/v3/projects/64092131887b4859ae47a1f4c3e0d24a => generated 298 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:57:52.828379 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b2fa7e-84a7-4a4a-a8b2-a4032bd16fe6 None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:57:52.856421 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 213/433] 10.222.0.23 () {58 vars in 1023 bytes} [Thu May 19 11:57:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 11:57:52.856421 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e8732f-6432-4dc0-a3e1-f1a81cbb2405 None None] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:57:52.956708 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b2fa7e-84a7-4a4a-a8b2-a4032bd16fe6 None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:57:52.969717 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e8732f-6432-4dc0-a3e1-f1a81cbb2405 None None] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:57:52.997700 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:57:53.004668 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fe9751-e2cb-4bbc-b95c-93729595a937 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:57:53.005503 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fe9751-e2cb-4bbc-b95c-93729595a937 None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:53.005855 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fe9751-e2cb-4bbc-b95c-93729595a937 None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:53.006172 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fe9751-e2cb-4bbc-b95c-93729595a937 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:53.013647 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f1a4bb-8828-4fe0-9fbd-2c66abe8450f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:57:53.014585 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f1a4bb-8828-4fe0-9fbd-2c66abe8450f None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:53.014921 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f1a4bb-8828-4fe0-9fbd-2c66abe8450f None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:53.015198 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f1a4bb-8828-4fe0-9fbd-2c66abe8450f None admin] PATH_INFO: `/v3/groups/5f71f3c2dd1d47f980fee625c78a2dee` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:53.019044 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f1a4bb-8828-4fe0-9fbd-2c66abe8450f None admin] RBAC: Authorizing `identity:get_group(group_id=5f71f3c2dd1d47f980fee625c78a2dee)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:57:53.019280 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66fe9751-e2cb-4bbc-b95c-93729595a937 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 11:57:53.020609 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f1a4bb-8828-4fe0-9fbd-2c66abe8450f None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:57:53.027322 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 214/434] 10.222.0.23 () {56 vars in 1180 bytes} [Thu May 19 11:57:52 2022] GET /identity/v3/groups/5f71f3c2dd1d47f980fee625c78a2dee => generated 226 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:57:53.027322 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f1a4bb-8828-4fe0-9fbd-2c66abe8450f None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:57:53.027650 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application [None req-66fe9751-e2cb-4bbc-b95c-93729595a937 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 11:57:53.027650 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 11:57:53.027650 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 11:57:53.027650 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 11:57:53.027650 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 11:57:53.027650 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 11:57:53.027650 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 11:57:53.027650 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 11:57:53.027650 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 11:57:53.027650 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 11:57:53.027650 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 11:57:53.027650 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 11:57:53.027650 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 11:57:53.027650 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 11:57:53.027650 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 11:57:53.027650 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 11:57:53.028674 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 11:57:53.028674 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 11:57:53.028674 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 11:57:53.028674 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 11:57:53.028674 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 11:57:53.028674 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 11:57:53.028674 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 11:57:53.028674 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 11:57:53.028674 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 11:57:53.028674 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.get_or_create( May 19 11:57:53.028674 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 11:57:53.028674 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application with Lock( May 19 11:57:53.028674 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 11:57:53.028674 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._enter() May 19 11:57:53.028674 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 11:57:53.028674 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 11:57:53.029493 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 11:57:53.029493 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.creator() May 19 11:57:53.029493 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 11:57:53.029493 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application created_value = creator( May 19 11:57:53.029493 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 11:57:53.029493 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 11:57:53.029493 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 11:57:53.029493 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 11:57:53.029493 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 11:57:53.029493 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 11:57:53.029493 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 11:57:53.029493 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application May 19 11:57:53.029493 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 221/435] 10.222.0.23 () {56 vars in 1114 bytes} [Thu May 19 11:57:52 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 11:57:53.031308 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fe9751-e2cb-4bbc-b95c-93729595a937 None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:57:53.057058 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:57:53.141669 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f1a4bb-8828-4fe0-9fbd-2c66abe8450f None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:57:53.154168 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fe9751-e2cb-4bbc-b95c-93729595a937 None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:57:53.176639 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d213949c-c718-4b48-98c4-0df741504c25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:57:53.177447 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d213949c-c718-4b48-98c4-0df741504c25 None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:53.177731 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d213949c-c718-4b48-98c4-0df741504c25 None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:53.178008 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d213949c-c718-4b48-98c4-0df741504c25 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:53.188175 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c89ec1-aa76-49f7-bf46-9ea148296a81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:57:53.188481 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d213949c-c718-4b48-98c4-0df741504c25 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 11:57:53.188826 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c89ec1-aa76-49f7-bf46-9ea148296a81 None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:53.189118 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c89ec1-aa76-49f7-bf46-9ea148296a81 None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:53.189401 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c89ec1-aa76-49f7-bf46-9ea148296a81 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:53.190229 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c89ec1-aa76-49f7-bf46-9ea148296a81 None admin] RBAC: Adding query filter params (group.id=5f71f3c2dd1d47f980fee625c78a2dee, role.id=259eb400cdb4439db72d14456ae9cd08, scope.project.id=64092131887b4859ae47a1f4c3e0d24a) {{(pid=80540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 11:57:53.190547 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c89ec1-aa76-49f7-bf46-9ea148296a81 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:57:53.192021 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c89ec1-aa76-49f7-bf46-9ea148296a81 None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:57:53.196768 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application [None req-d213949c-c718-4b48-98c4-0df741504c25 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 11:57:53.196768 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 11:57:53.196768 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 11:57:53.196768 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 11:57:53.196768 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 11:57:53.196768 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 11:57:53.196768 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 11:57:53.196768 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 11:57:53.196768 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 11:57:53.196768 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 11:57:53.196768 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 11:57:53.196768 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 11:57:53.196768 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 11:57:53.196768 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 11:57:53.196768 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 11:57:53.196768 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 11:57:53.197631 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 11:57:53.197631 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 11:57:53.197631 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 11:57:53.197631 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 11:57:53.197631 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 11:57:53.197631 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 11:57:53.197631 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 11:57:53.197631 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 11:57:53.197631 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 11:57:53.197631 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.get_or_create( May 19 11:57:53.197631 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 11:57:53.197631 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application with Lock( May 19 11:57:53.197631 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 11:57:53.197631 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._enter() May 19 11:57:53.197631 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 11:57:53.197631 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 11:57:53.198416 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 11:57:53.198416 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.creator() May 19 11:57:53.198416 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 11:57:53.198416 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application created_value = creator( May 19 11:57:53.198416 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 11:57:53.198416 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 11:57:53.198416 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 11:57:53.198416 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 11:57:53.198416 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 11:57:53.198416 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 11:57:53.198416 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 11:57:53.198416 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application May 19 11:57:53.198416 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 215/436] 10.222.0.23 () {56 vars in 1114 bytes} [Thu May 19 11:57:53 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 11:57:53.199196 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 222/437] 10.222.0.23 () {56 vars in 1376 bytes} [Thu May 19 11:57:53 2022] GET /identity/v3/role_assignments?group.id=5f71f3c2dd1d47f980fee625c78a2dee&scope.project.id=64092131887b4859ae47a1f4c3e0d24a&role.id=259eb400cdb4439db72d14456ae9cd08 => generated 611 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:57:53.199262 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d213949c-c718-4b48-98c4-0df741504c25 None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:57:53.199325 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e1520c-3739-4946-b981-37b3a5429da1 None None] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:53.199463 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e1520c-3739-4946-b981-37b3a5429da1 None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:53.199775 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e1520c-3739-4946-b981-37b3a5429da1 None None] PATH_INFO: `/` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:53.201011 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 223/438] 10.222.0.23 () {54 vars in 926 bytes} [Thu May 19 11:57:53 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 11:57:53.205343 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e1520c-3739-4946-b981-37b3a5429da1 None None] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:57:53.274304 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:57:53.306632 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:57:53.329144 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d213949c-c718-4b48-98c4-0df741504c25 None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:57:53.331225 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e1520c-3739-4946-b981-37b3a5429da1 None None] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:57:53.366344 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fd3c93-64ff-4bfd-b2e8-5d476021dbde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:57:53.367035 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fd3c93-64ff-4bfd-b2e8-5d476021dbde None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:53.367329 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fd3c93-64ff-4bfd-b2e8-5d476021dbde None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:53.367612 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fd3c93-64ff-4bfd-b2e8-5d476021dbde None admin] PATH_INFO: `/v3/roles` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:53.368426 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6fd3c93-64ff-4bfd-b2e8-5d476021dbde None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 11:57:53.368769 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6fd3c93-64ff-4bfd-b2e8-5d476021dbde None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:57:53.368979 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2849d3-9876-4e18-bdb7-947e46d986e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:57:53.369658 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2849d3-9876-4e18-bdb7-947e46d986e0 None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:53.369943 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2849d3-9876-4e18-bdb7-947e46d986e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:53.370190 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6fd3c93-64ff-4bfd-b2e8-5d476021dbde None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:57:53.370282 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2849d3-9876-4e18-bdb7-947e46d986e0 None admin] PATH_INFO: `/v3/roles` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:53.371004 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2849d3-9876-4e18-bdb7-947e46d986e0 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 11:57:53.371287 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2849d3-9876-4e18-bdb7-947e46d986e0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:57:53.372749 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2849d3-9876-4e18-bdb7-947e46d986e0 None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:57:53.380637 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 216/439] 10.222.0.23 () {56 vars in 1111 bytes} [Thu May 19 11:57:53 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:57:53.381936 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f83c0c-c6c0-4a41-8953-2361b62703e5 None None] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:53.382235 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f83c0c-c6c0-4a41-8953-2361b62703e5 None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:53.382516 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f83c0c-c6c0-4a41-8953-2361b62703e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:53.387828 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 224/440] 10.222.0.23 () {56 vars in 1111 bytes} [Thu May 19 11:57:53 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:57:53.387828 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2849d3-9876-4e18-bdb7-947e46d986e0 None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:57:53.406448 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.auth.core [None req-49f83c0c-c6c0-4a41-8953-2361b62703e5 None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 11:57:53.407753 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-49f83c0c-c6c0-4a41-8953-2361b62703e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 11:57:53.500040 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 217/441] 10.222.0.23 () {58 vars in 1023 bytes} [Thu May 19 11:57:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 11:57:53.500842 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f83c0c-c6c0-4a41-8953-2361b62703e5 None None] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:57:53.530772 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2849d3-9876-4e18-bdb7-947e46d986e0 None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:57:53.548183 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:57:53.574165 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7226ce71-baef-4246-97ce-2d1b9dc0128d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:57:53.574822 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7226ce71-baef-4246-97ce-2d1b9dc0128d None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:53.575112 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7226ce71-baef-4246-97ce-2d1b9dc0128d None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:53.575394 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7226ce71-baef-4246-97ce-2d1b9dc0128d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:53.588281 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:57:53.626511 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f83c0c-c6c0-4a41-8953-2361b62703e5 None None] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:57:53.655012 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7226ce71-baef-4246-97ce-2d1b9dc0128d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:57:53.656535 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7226ce71-baef-4246-97ce-2d1b9dc0128d None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:57:53.661227 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91ff766-a719-4ebf-82f4-0bdd69769eca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:57:53.661880 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91ff766-a719-4ebf-82f4-0bdd69769eca None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:53.662168 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91ff766-a719-4ebf-82f4-0bdd69769eca None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:53.662446 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91ff766-a719-4ebf-82f4-0bdd69769eca None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:53.732238 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91ff766-a719-4ebf-82f4-0bdd69769eca None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:57:53.734332 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91ff766-a719-4ebf-82f4-0bdd69769eca None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:57:53.763392 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 225/442] 10.222.0.23 () {58 vars in 1282 bytes} [Thu May 19 11:57:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 11:57:53.764881 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7226ce71-baef-4246-97ce-2d1b9dc0128d None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:57:53.784159 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:57:53.808056 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:57:53.840208 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 218/443] 10.222.0.23 () {58 vars in 1282 bytes} [Thu May 19 11:57:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 11:57:53.841064 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91ff766-a719-4ebf-82f4-0bdd69769eca None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:57:53.876921 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7226ce71-baef-4246-97ce-2d1b9dc0128d None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:57:53.910383 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad684dc6-cf58-4a0e-ad33-b980b8f7765e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:57:53.911200 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad684dc6-cf58-4a0e-ad33-b980b8f7765e None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:53.911482 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad684dc6-cf58-4a0e-ad33-b980b8f7765e None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:53.911757 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad684dc6-cf58-4a0e-ad33-b980b8f7765e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:53.922435 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad684dc6-cf58-4a0e-ad33-b980b8f7765e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 11:57:53.930253 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application [None req-ad684dc6-cf58-4a0e-ad33-b980b8f7765e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 11:57:53.930253 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 11:57:53.930253 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 11:57:53.930253 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 11:57:53.930253 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 11:57:53.930253 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 11:57:53.930253 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 11:57:53.930253 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 11:57:53.930253 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 11:57:53.930253 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 11:57:53.930253 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 11:57:53.930253 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 11:57:53.930253 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 11:57:53.930253 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 11:57:53.930253 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 11:57:53.930253 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 11:57:53.931152 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 11:57:53.931152 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 11:57:53.931152 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 11:57:53.931152 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 11:57:53.931152 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 11:57:53.931152 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 11:57:53.931152 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 11:57:53.931152 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 11:57:53.931152 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 11:57:53.931152 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.get_or_create( May 19 11:57:53.931152 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 11:57:53.931152 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application with Lock( May 19 11:57:53.931152 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 11:57:53.931152 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._enter() May 19 11:57:53.931152 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 11:57:53.931152 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 11:57:53.931985 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 11:57:53.931985 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.creator() May 19 11:57:53.931985 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 11:57:53.931985 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application created_value = creator( May 19 11:57:53.931985 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 11:57:53.931985 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 11:57:53.931985 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 11:57:53.931985 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 11:57:53.931985 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 11:57:53.931985 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 11:57:53.931985 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 11:57:53.931985 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application May 19 11:57:53.931985 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 226/444] 10.222.0.23 () {56 vars in 1096 bytes} [Thu May 19 11:57:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 11:57:53.932826 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad684dc6-cf58-4a0e-ad33-b980b8f7765e None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:57:53.949520 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91ff766-a719-4ebf-82f4-0bdd69769eca None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:57:53.984919 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d231ff0-e993-4cab-ac82-3f5b26467b99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:57:53.985888 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d231ff0-e993-4cab-ac82-3f5b26467b99 None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:53.986243 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d231ff0-e993-4cab-ac82-3f5b26467b99 None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:53.986579 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d231ff0-e993-4cab-ac82-3f5b26467b99 None admin] PATH_INFO: `/v3/projects/64092131887b4859ae47a1f4c3e0d24a` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:53.987900 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d231ff0-e993-4cab-ac82-3f5b26467b99 None admin] RBAC: Authorizing `identity:get_project(project_id=64092131887b4859ae47a1f4c3e0d24a)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:57:53.989393 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d231ff0-e993-4cab-ac82-3f5b26467b99 None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:57:53.991092 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 219/445] 10.222.0.23 () {56 vars in 1186 bytes} [Thu May 19 11:57:53 2022] GET /identity/v3/projects/64092131887b4859ae47a1f4c3e0d24a => generated 298 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:57:53.991882 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d231ff0-e993-4cab-ac82-3f5b26467b99 None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:57:54.035221 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:57:54.035777 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:57:54.057747 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad684dc6-cf58-4a0e-ad33-b980b8f7765e None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:57:54.093691 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb146ae-b694-4ff6-b555-a875fd01baa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:57:54.093911 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d231ff0-e993-4cab-ac82-3f5b26467b99 None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:57:54.094546 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb146ae-b694-4ff6-b555-a875fd01baa2 None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:54.094835 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb146ae-b694-4ff6-b555-a875fd01baa2 None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:54.095116 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb146ae-b694-4ff6-b555-a875fd01baa2 None admin] PATH_INFO: `/v3/projects/0c5d68529f48460f9f6e03e3f8fb205b` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:54.096333 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb146ae-b694-4ff6-b555-a875fd01baa2 None admin] RBAC: Authorizing `identity:get_project(project_id=0c5d68529f48460f9f6e03e3f8fb205b)` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:57:54.097794 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb146ae-b694-4ff6-b555-a875fd01baa2 None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:57:54.099488 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 227/446] 10.222.0.23 () {56 vars in 1186 bytes} [Thu May 19 11:57:53 2022] GET /identity/v3/projects/0c5d68529f48460f9f6e03e3f8fb205b => generated 302 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:57:54.100465 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb146ae-b694-4ff6-b555-a875fd01baa2 None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:57:54.131564 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6496c98d-7f57-4e4a-90ec-798f6f62a49d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:57:54.132265 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6496c98d-7f57-4e4a-90ec-798f6f62a49d None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:54.132567 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6496c98d-7f57-4e4a-90ec-798f6f62a49d None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:54.132835 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6496c98d-7f57-4e4a-90ec-798f6f62a49d None admin] PATH_INFO: `/v3/roles` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:54.133586 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6496c98d-7f57-4e4a-90ec-798f6f62a49d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 11:57:54.133897 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6496c98d-7f57-4e4a-90ec-798f6f62a49d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:57:54.135282 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6496c98d-7f57-4e4a-90ec-798f6f62a49d None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:57:54.144574 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 220/447] 10.222.0.23 () {56 vars in 1099 bytes} [Thu May 19 11:57:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:57:54.145413 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6496c98d-7f57-4e4a-90ec-798f6f62a49d None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:57:54.204506 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb146ae-b694-4ff6-b555-a875fd01baa2 None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:57:54.238879 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95bd30cb-fc09-4251-93ab-703de0a2423d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:57:54.239753 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bd30cb-fc09-4251-93ab-703de0a2423d None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:54.240073 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bd30cb-fc09-4251-93ab-703de0a2423d None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:54.240351 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bd30cb-fc09-4251-93ab-703de0a2423d None admin] PATH_INFO: `/v3/groups/5f71f3c2dd1d47f980fee625c78a2dee` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:54.241555 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95bd30cb-fc09-4251-93ab-703de0a2423d None admin] RBAC: Authorizing `identity:get_group(group_id=5f71f3c2dd1d47f980fee625c78a2dee)` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:57:54.243026 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95bd30cb-fc09-4251-93ab-703de0a2423d None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:57:54.244699 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 228/448] 10.222.0.23 () {56 vars in 1180 bytes} [Thu May 19 11:57:54 2022] GET /identity/v3/groups/5f71f3c2dd1d47f980fee625c78a2dee => generated 226 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:57:54.245465 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95bd30cb-fc09-4251-93ab-703de0a2423d None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:57:54.246923 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6496c98d-7f57-4e4a-90ec-798f6f62a49d None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:57:54.249503 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:57:54.263268 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:57:54.294936 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bb83a4-4dc5-46c6-86e9-aefcbcf13d26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:57:54.295938 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bb83a4-4dc5-46c6-86e9-aefcbcf13d26 None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:54.296224 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bb83a4-4dc5-46c6-86e9-aefcbcf13d26 None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:54.296501 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bb83a4-4dc5-46c6-86e9-aefcbcf13d26 None admin] PATH_INFO: `/v3/groups/5f71f3c2dd1d47f980fee625c78a2dee` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:54.297743 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70bb83a4-4dc5-46c6-86e9-aefcbcf13d26 None admin] RBAC: Authorizing `identity:get_group(group_id=5f71f3c2dd1d47f980fee625c78a2dee)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:57:54.299210 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70bb83a4-4dc5-46c6-86e9-aefcbcf13d26 None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:57:54.300897 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 221/449] 10.222.0.23 () {56 vars in 1180 bytes} [Thu May 19 11:57:54 2022] GET /identity/v3/groups/5f71f3c2dd1d47f980fee625c78a2dee => generated 226 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:57:54.301679 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bb83a4-4dc5-46c6-86e9-aefcbcf13d26 None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:57:54.360938 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95bd30cb-fc09-4251-93ab-703de0a2423d None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:57:54.397611 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0d13a6-5c1a-4bb2-a145-16c1257aabbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:57:54.398252 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0d13a6-5c1a-4bb2-a145-16c1257aabbe None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:54.398536 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0d13a6-5c1a-4bb2-a145-16c1257aabbe None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:54.398807 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0d13a6-5c1a-4bb2-a145-16c1257aabbe None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:54.405774 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bb83a4-4dc5-46c6-86e9-aefcbcf13d26 None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:57:54.440118 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed19b7b6-64ca-4331-8ce6-18fff04eb3ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:57:54.440770 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed19b7b6-64ca-4331-8ce6-18fff04eb3ee None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:54.441053 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed19b7b6-64ca-4331-8ce6-18fff04eb3ee None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:54.441329 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed19b7b6-64ca-4331-8ce6-18fff04eb3ee None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:54.442170 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed19b7b6-64ca-4331-8ce6-18fff04eb3ee None admin] RBAC: Adding query filter params (group.id=5f71f3c2dd1d47f980fee625c78a2dee, role.id=8215583bb4224c75b093ae8e3993eb9f, scope.project.id=64092131887b4859ae47a1f4c3e0d24a) {{(pid=80539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 11:57:54.442477 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed19b7b6-64ca-4331-8ce6-18fff04eb3ee None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:57:54.443935 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed19b7b6-64ca-4331-8ce6-18fff04eb3ee None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:57:54.450127 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 222/450] 10.222.0.23 () {56 vars in 1376 bytes} [Thu May 19 11:57:54 2022] GET /identity/v3/role_assignments?group.id=5f71f3c2dd1d47f980fee625c78a2dee&scope.project.id=64092131887b4859ae47a1f4c3e0d24a&role.id=8215583bb4224c75b093ae8e3993eb9f => generated 611 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:57:54.450735 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed19b7b6-64ca-4331-8ce6-18fff04eb3ee None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:57:54.477980 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0d13a6-5c1a-4bb2-a145-16c1257aabbe None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:57:54.479656 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0d13a6-5c1a-4bb2-a145-16c1257aabbe None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:57:54.492618 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:57:54.496676 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:57:54.572308 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed19b7b6-64ca-4331-8ce6-18fff04eb3ee None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:57:54.606184 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 229/451] 10.222.0.23 () {58 vars in 1282 bytes} [Thu May 19 11:57:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 11:57:54.612341 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0d13a6-5c1a-4bb2-a145-16c1257aabbe None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:57:54.614203 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f492633-8979-43a9-8210-aedca6d26b42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:57:54.614992 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f492633-8979-43a9-8210-aedca6d26b42 None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:54.615276 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f492633-8979-43a9-8210-aedca6d26b42 None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:54.615553 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f492633-8979-43a9-8210-aedca6d26b42 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:54.616444 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f492633-8979-43a9-8210-aedca6d26b42 None admin] RBAC: Adding query filter params (group.id=5f71f3c2dd1d47f980fee625c78a2dee, role.id=8215583bb4224c75b093ae8e3993eb9f, scope.project.id=0c5d68529f48460f9f6e03e3f8fb205b) {{(pid=80539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 11:57:54.616761 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f492633-8979-43a9-8210-aedca6d26b42 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:57:54.618279 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f492633-8979-43a9-8210-aedca6d26b42 None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:57:54.625555 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 223/452] 10.222.0.23 () {56 vars in 1376 bytes} [Thu May 19 11:57:54 2022] GET /identity/v3/role_assignments?group.id=5f71f3c2dd1d47f980fee625c78a2dee&scope.project.id=0c5d68529f48460f9f6e03e3f8fb205b&role.id=8215583bb4224c75b093ae8e3993eb9f => generated 611 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:57:54.763125 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0d13a6-5c1a-4bb2-a145-16c1257aabbe None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:57:54.779020 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:57:54.804702 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7921ae94-7256-46d6-a014-d1591aaecdb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:57:54.805854 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7921ae94-7256-46d6-a014-d1591aaecdb4 None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:54.806419 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7921ae94-7256-46d6-a014-d1591aaecdb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:54.806701 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7921ae94-7256-46d6-a014-d1591aaecdb4 None admin] PATH_INFO: `/v3/projects/f3569780f2fe4b24b4109e2eee3c26fe` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:54.807974 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7921ae94-7256-46d6-a014-d1591aaecdb4 None admin] RBAC: Authorizing `identity:get_project(project_id=f3569780f2fe4b24b4109e2eee3c26fe)` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:57:54.809453 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7921ae94-7256-46d6-a014-d1591aaecdb4 None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:57:54.811700 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 230/453] 10.222.0.23 () {56 vars in 1186 bytes} [Thu May 19 11:57:54 2022] GET /identity/v3/projects/f3569780f2fe4b24b4109e2eee3c26fe => generated 344 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:57:54.817296 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f492633-8979-43a9-8210-aedca6d26b42 None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:57:54.926038 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:57:54.932039 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f492633-8979-43a9-8210-aedca6d26b42 None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:57:54.969188 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7e05c6-667e-4410-924b-23a767b22603 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:57:54.970223 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7e05c6-667e-4410-924b-23a767b22603 None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:54.970566 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7e05c6-667e-4410-924b-23a767b22603 None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:54.971144 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7e05c6-667e-4410-924b-23a767b22603 None admin] PATH_INFO: `/v3/groups/2b83c34e4dc54b2fb1afe8a56166cd2c` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:54.972343 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7e05c6-667e-4410-924b-23a767b22603 None admin] RBAC: Authorizing `identity:get_group(group_id=2b83c34e4dc54b2fb1afe8a56166cd2c)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:57:54.973768 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7e05c6-667e-4410-924b-23a767b22603 None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:57:54.975520 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 224/454] 10.222.0.23 () {56 vars in 1180 bytes} [Thu May 19 11:57:54 2022] GET /identity/v3/groups/2b83c34e4dc54b2fb1afe8a56166cd2c => generated 229 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:57:54.981578 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7921ae94-7256-46d6-a014-d1591aaecdb4 None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:57:55.105179 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7921ae94-7256-46d6-a014-d1591aaecdb4 None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:57:55.140459 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3174afb-63a7-4474-83bf-1395c6fdf2ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:57:55.140999 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3174afb-63a7-4474-83bf-1395c6fdf2ed None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:55.141309 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3174afb-63a7-4474-83bf-1395c6fdf2ed None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:55.141586 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3174afb-63a7-4474-83bf-1395c6fdf2ed None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:55.142479 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3174afb-63a7-4474-83bf-1395c6fdf2ed None admin] RBAC: Adding query filter params (group.id=2b83c34e4dc54b2fb1afe8a56166cd2c, role.id=9db5671c65d3414ea2682afa2ccf1403, scope.project.id=f3569780f2fe4b24b4109e2eee3c26fe) {{(pid=80540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 11:57:55.142822 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3174afb-63a7-4474-83bf-1395c6fdf2ed None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:57:55.144377 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3174afb-63a7-4474-83bf-1395c6fdf2ed None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:57:55.150625 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 231/455] 10.222.0.23 () {56 vars in 1376 bytes} [Thu May 19 11:57:54 2022] GET /identity/v3/role_assignments?group.id=2b83c34e4dc54b2fb1afe8a56166cd2c&scope.project.id=f3569780f2fe4b24b4109e2eee3c26fe&role.id=9db5671c65d3414ea2682afa2ccf1403 => generated 611 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:57:58.003256 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cb03bd-6093-4444-b513-10a96a04def7 None None] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:58.003950 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cb03bd-6093-4444-b513-10a96a04def7 None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:58.003950 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cb03bd-6093-4444-b513-10a96a04def7 None None] PATH_INFO: `/` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:58.005478 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 225/456] 10.222.0.23 () {54 vars in 926 bytes} [Thu May 19 11:57:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 11:57:58.010642 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f094315d-90ce-4925-ac94-29860396c4f0 None None] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:58.011414 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f094315d-90ce-4925-ac94-29860396c4f0 None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:58.011869 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f094315d-90ce-4925-ac94-29860396c4f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:58.036762 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.auth.core [None req-f094315d-90ce-4925-ac94-29860396c4f0 None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 11:57:58.038215 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-f094315d-90ce-4925-ac94-29860396c4f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 11:57:58.094229 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:57:58.129448 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 232/457] 10.222.0.23 () {58 vars in 1023 bytes} [Thu May 19 11:57:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 11:57:58.164224 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec5873c-4e54-4b07-b68d-609b3bd996d7 None None] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:58.164880 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec5873c-4e54-4b07-b68d-609b3bd996d7 None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:58.165552 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec5873c-4e54-4b07-b68d-609b3bd996d7 None None] PATH_INFO: `/` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:58.166172 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f094315d-90ce-4925-ac94-29860396c4f0 None None] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:57:58.167596 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 226/458] 10.222.0.23 () {54 vars in 926 bytes} [Thu May 19 11:57:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 11:57:58.174292 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e933a60-5b83-44c5-aedd-78638e3745a3 None None] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:58.174895 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e933a60-5b83-44c5-aedd-78638e3745a3 None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:58.175439 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e933a60-5b83-44c5-aedd-78638e3745a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:58.206987 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.auth.core [None req-3e933a60-5b83-44c5-aedd-78638e3745a3 None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 11:57:58.208629 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-3e933a60-5b83-44c5-aedd-78638e3745a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 11:57:58.329040 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f094315d-90ce-4925-ac94-29860396c4f0 None None] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:57:58.334319 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 227/459] 10.222.0.23 () {58 vars in 1023 bytes} [Thu May 19 11:57:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 11:57:58.336036 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de31941-046c-4200-86df-acdc3c413a99 None None] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:58.336467 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de31941-046c-4200-86df-acdc3c413a99 None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:58.336883 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de31941-046c-4200-86df-acdc3c413a99 None None] PATH_INFO: `/` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:58.342926 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 228/460] 10.222.0.23 () {54 vars in 926 bytes} [Thu May 19 11:57:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 11:57:58.344792 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f61c29b-1b5e-46f0-b088-28ece718ad79 None None] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:58.345316 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f61c29b-1b5e-46f0-b088-28ece718ad79 None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:58.345728 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f61c29b-1b5e-46f0-b088-28ece718ad79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:58.370545 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.auth.core [None req-6f61c29b-1b5e-46f0-b088-28ece718ad79 None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 11:57:58.371870 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-6f61c29b-1b5e-46f0-b088-28ece718ad79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 11:57:58.379863 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c01d6d-70d9-4e6c-8a44-9318ae8cd204 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:57:58.381011 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c01d6d-70d9-4e6c-8a44-9318ae8cd204 None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:58.381448 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c01d6d-70d9-4e6c-8a44-9318ae8cd204 None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:58.381864 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c01d6d-70d9-4e6c-8a44-9318ae8cd204 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:58.393614 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c01d6d-70d9-4e6c-8a44-9318ae8cd204 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:57:58.395438 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c01d6d-70d9-4e6c-8a44-9318ae8cd204 None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:57:58.416189 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application [None req-63c01d6d-70d9-4e6c-8a44-9318ae8cd204 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 11:57:58.416189 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 11:57:58.416189 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 11:57:58.416189 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 11:57:58.416189 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 11:57:58.416189 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 11:57:58.416189 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 11:57:58.416189 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 11:57:58.416189 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 11:57:58.416189 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 11:57:58.416189 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 11:57:58.416189 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 11:57:58.416189 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 11:57:58.416189 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 11:57:58.416189 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 11:57:58.417273 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 11:57:58.417273 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 11:57:58.417273 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 11:57:58.417273 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 11:57:58.417273 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 11:57:58.417273 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 11:57:58.417273 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.get_or_create( May 19 11:57:58.417273 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 11:57:58.417273 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application with Lock( May 19 11:57:58.417273 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 11:57:58.417273 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._enter() May 19 11:57:58.417273 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 11:57:58.417273 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 11:57:58.417273 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 11:57:58.417273 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.creator() May 19 11:57:58.417273 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 11:57:58.417273 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application created_value = creator( May 19 11:57:58.418276 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 11:57:58.418276 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 11:57:58.418276 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 11:57:58.418276 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application May 19 11:57:58.419927 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 233/461] 10.222.0.23 () {56 vars in 1108 bytes} [Thu May 19 11:57:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 11:57:58.421695 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c01d6d-70d9-4e6c-8a44-9318ae8cd204 None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:57:58.452787 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 229/462] 10.222.0.23 () {58 vars in 1023 bytes} [Thu May 19 11:57:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 11:57:58.459906 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f61c29b-1b5e-46f0-b088-28ece718ad79 None None] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:57:58.460550 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:57:58.484997 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:57:58.562172 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c01d6d-70d9-4e6c-8a44-9318ae8cd204 None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:57:58.565472 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f61c29b-1b5e-46f0-b088-28ece718ad79 None None] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:57:58.599857 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345c68b9-d3bb-437b-a233-e42b3107f19b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:57:58.601092 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345c68b9-d3bb-437b-a233-e42b3107f19b None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:58.601525 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345c68b9-d3bb-437b-a233-e42b3107f19b None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:58.601948 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345c68b9-d3bb-437b-a233-e42b3107f19b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:58.602138 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22fa48d-ae32-47c8-bed8-928b367dfcdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:57:58.602806 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22fa48d-ae32-47c8-bed8-928b367dfcdd None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:58.603088 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22fa48d-ae32-47c8-bed8-928b367dfcdd None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:58.603363 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22fa48d-ae32-47c8-bed8-928b367dfcdd None admin] PATH_INFO: `/v3/domains` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:58.604132 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22fa48d-ae32-47c8-bed8-928b367dfcdd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 11:57:58.604446 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22fa48d-ae32-47c8-bed8-928b367dfcdd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:57:58.605815 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22fa48d-ae32-47c8-bed8-928b367dfcdd None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:57:58.616579 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345c68b9-d3bb-437b-a233-e42b3107f19b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:57:58.618262 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345c68b9-d3bb-437b-a233-e42b3107f19b None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:57:58.618953 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 230/463] 10.222.0.23 () {56 vars in 1109 bytes} [Thu May 19 11:57:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:57:58.619776 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22fa48d-ae32-47c8-bed8-928b367dfcdd None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:57:58.640792 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application [None req-345c68b9-d3bb-437b-a233-e42b3107f19b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 11:57:58.640792 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 11:57:58.640792 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 11:57:58.640792 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 11:57:58.640792 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 11:57:58.640792 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 11:57:58.640792 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 11:57:58.640792 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 11:57:58.640792 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 11:57:58.640792 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 11:57:58.640792 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 11:57:58.640792 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 11:57:58.640792 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 11:57:58.640792 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 11:57:58.640792 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 11:57:58.641837 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 11:57:58.641837 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 11:57:58.641837 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 11:57:58.641837 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 11:57:58.641837 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 11:57:58.641837 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 11:57:58.641837 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.get_or_create( May 19 11:57:58.641837 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 11:57:58.641837 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application with Lock( May 19 11:57:58.641837 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 11:57:58.641837 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._enter() May 19 11:57:58.641837 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 11:57:58.641837 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 11:57:58.641837 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 11:57:58.641837 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.creator() May 19 11:57:58.641837 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 11:57:58.641837 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application created_value = creator( May 19 11:57:58.642869 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 11:57:58.642869 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 11:57:58.642869 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 11:57:58.642869 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application May 19 11:57:58.644487 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 234/464] 10.222.0.23 () {56 vars in 1108 bytes} [Thu May 19 11:57:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 11:57:58.645790 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee72529-400d-40b3-b486-a9d18c7b18d7 None None] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:58.646086 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee72529-400d-40b3-b486-a9d18c7b18d7 None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:58.646367 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee72529-400d-40b3-b486-a9d18c7b18d7 None None] PATH_INFO: `/` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:58.651825 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 235/465] 10.222.0.23 () {54 vars in 926 bytes} [Thu May 19 11:57:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 11:57:58.651825 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee72529-400d-40b3-b486-a9d18c7b18d7 None None] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:57:58.728217 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:57:58.736869 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:57:58.747492 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22fa48d-ae32-47c8-bed8-928b367dfcdd None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:57:58.761972 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee72529-400d-40b3-b486-a9d18c7b18d7 None None] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:57:58.788116 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a719655-b48d-4c84-aaa5-86d01b586504 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:57:58.789751 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a719655-b48d-4c84-aaa5-86d01b586504 None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:58.790220 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a719655-b48d-4c84-aaa5-86d01b586504 None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:58.790688 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a719655-b48d-4c84-aaa5-86d01b586504 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:58.805074 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a719655-b48d-4c84-aaa5-86d01b586504 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:57:58.805374 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a313c04-aa8d-4c51-bc5d-c56adfaf6542 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:57:58.805984 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a313c04-aa8d-4c51-bc5d-c56adfaf6542 None admin] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:58.806273 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a313c04-aa8d-4c51-bc5d-c56adfaf6542 None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:58.806534 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a719655-b48d-4c84-aaa5-86d01b586504 None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:57:58.806628 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a313c04-aa8d-4c51-bc5d-c56adfaf6542 None admin] PATH_INFO: `/v3/users` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:58.807245 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a313c04-aa8d-4c51-bc5d-c56adfaf6542 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:57:58.808729 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a313c04-aa8d-4c51-bc5d-c56adfaf6542 None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:57:58.825547 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application [None req-2a719655-b48d-4c84-aaa5-86d01b586504 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 11:57:58.825547 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 11:57:58.825547 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 11:57:58.825547 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 11:57:58.825547 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 11:57:58.825547 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 11:57:58.825547 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 11:57:58.825547 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 11:57:58.825547 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 11:57:58.825547 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 11:57:58.825547 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 11:57:58.825547 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 11:57:58.825547 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 11:57:58.825547 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 11:57:58.825547 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 11:57:58.826647 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 11:57:58.826647 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 11:57:58.826647 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 11:57:58.826647 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 11:57:58.826647 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 11:57:58.826647 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 11:57:58.826647 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.get_or_create( May 19 11:57:58.826647 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 11:57:58.826647 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application with Lock( May 19 11:57:58.826647 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 11:57:58.826647 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._enter() May 19 11:57:58.826647 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 11:57:58.826647 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 11:57:58.826647 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 11:57:58.826647 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.creator() May 19 11:57:58.826647 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 11:57:58.826647 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application created_value = creator( May 19 11:57:58.827531 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 11:57:58.827531 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 11:57:58.827531 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 11:57:58.827531 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application May 19 11:57:58.827531 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 231/466] 10.222.0.23 () {56 vars in 1108 bytes} [Thu May 19 11:57:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 11:57:58.827800 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a719655-b48d-4c84-aaa5-86d01b586504 None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:57:58.908556 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 236/467] 10.222.0.23 () {60 vars in 1132 bytes} [Thu May 19 11:57:58 2022] POST /identity/v3/users => generated 246 bytes in 260 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 11:57:58.916027 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b6f620-6b3a-4b6e-a346-735a2b4b365f None None] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:58.916339 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b6f620-6b3a-4b6e-a346-735a2b4b365f None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:58.916603 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b6f620-6b3a-4b6e-a346-735a2b4b365f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:58.961345 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a719655-b48d-4c84-aaa5-86d01b586504 None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:57:58.966293 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.auth.core [None req-27b6f620-6b3a-4b6e-a346-735a2b4b365f None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 11:57:58.968613 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-27b6f620-6b3a-4b6e-a346-735a2b4b365f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 11:57:59.009515 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3712fa4f-19f1-43db-875e-eb76094ee364 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:57:59.010200 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3712fa4f-19f1-43db-875e-eb76094ee364 None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:59.010381 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3712fa4f-19f1-43db-875e-eb76094ee364 None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:59.010664 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3712fa4f-19f1-43db-875e-eb76094ee364 None admin] PATH_INFO: `/v3/domains` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:59.011890 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3712fa4f-19f1-43db-875e-eb76094ee364 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 11:57:59.012211 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3712fa4f-19f1-43db-875e-eb76094ee364 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:57:59.013943 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3712fa4f-19f1-43db-875e-eb76094ee364 None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:57:59.025286 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:57:59.031533 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 232/468] 10.222.0.23 () {56 vars in 1109 bytes} [Thu May 19 11:57:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:57:59.032472 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3712fa4f-19f1-43db-875e-eb76094ee364 None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:57:59.068131 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 237/469] 10.222.0.23 () {58 vars in 1023 bytes} [Thu May 19 11:57:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 11:57:59.070083 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b6f620-6b3a-4b6e-a346-735a2b4b365f None None] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:57:59.160071 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:57:59.195178 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3712fa4f-19f1-43db-875e-eb76094ee364 None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:57:59.228672 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b6f620-6b3a-4b6e-a346-735a2b4b365f None None] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:57:59.251093 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9530de-b5a4-4e26-aced-5710841acf11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:57:59.252067 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9530de-b5a4-4e26-aced-5710841acf11 None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:59.252403 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9530de-b5a4-4e26-aced-5710841acf11 None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:59.252849 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9530de-b5a4-4e26-aced-5710841acf11 None admin] PATH_INFO: `/v3/domains` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:59.253746 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9530de-b5a4-4e26-aced-5710841acf11 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 11:57:59.254256 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9530de-b5a4-4e26-aced-5710841acf11 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:57:59.257946 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9530de-b5a4-4e26-aced-5710841acf11 None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:57:59.275989 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759f0913-c950-42b5-b3b8-8cb41cd088b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:57:59.276287 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 233/470] 10.222.0.23 () {56 vars in 1109 bytes} [Thu May 19 11:57:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:57:59.276662 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759f0913-c950-42b5-b3b8-8cb41cd088b5 None admin] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:59.276956 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759f0913-c950-42b5-b3b8-8cb41cd088b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:59.277150 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9530de-b5a4-4e26-aced-5710841acf11 None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:57:59.277248 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759f0913-c950-42b5-b3b8-8cb41cd088b5 None admin] PATH_INFO: `/v3/users` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:59.277967 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759f0913-c950-42b5-b3b8-8cb41cd088b5 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:57:59.279500 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759f0913-c950-42b5-b3b8-8cb41cd088b5 None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:57:59.349316 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:57:59.366156 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 238/471] 10.222.0.23 () {60 vars in 1132 bytes} [Thu May 19 11:57:59 2022] POST /identity/v3/users => generated 248 bytes in 297 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 11:57:59.369708 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759f0913-c950-42b5-b3b8-8cb41cd088b5 None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:57:59.436478 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9530de-b5a4-4e26-aced-5710841acf11 None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:57:59.497363 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c93a386-a8f8-49fb-8f0c-412d18118ab1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:57:59.498175 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c93a386-a8f8-49fb-8f0c-412d18118ab1 None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:59.498992 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c93a386-a8f8-49fb-8f0c-412d18118ab1 None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:59.499271 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c93a386-a8f8-49fb-8f0c-412d18118ab1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:59.514759 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c93a386-a8f8-49fb-8f0c-412d18118ab1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:57:59.517089 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c93a386-a8f8-49fb-8f0c-412d18118ab1 None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:57:59.529001 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759f0913-c950-42b5-b3b8-8cb41cd088b5 None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:57:59.540884 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application [None req-0c93a386-a8f8-49fb-8f0c-412d18118ab1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 11:57:59.540884 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 11:57:59.540884 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 11:57:59.540884 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 11:57:59.540884 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 11:57:59.540884 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 11:57:59.540884 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 11:57:59.540884 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 11:57:59.540884 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 11:57:59.540884 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 11:57:59.540884 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 11:57:59.540884 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 11:57:59.540884 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 11:57:59.540884 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 11:57:59.540884 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 11:57:59.541813 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 11:57:59.541813 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 11:57:59.541813 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 11:57:59.541813 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 11:57:59.541813 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 11:57:59.541813 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 11:57:59.541813 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.get_or_create( May 19 11:57:59.541813 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 11:57:59.541813 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application with Lock( May 19 11:57:59.541813 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 11:57:59.541813 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._enter() May 19 11:57:59.541813 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 11:57:59.541813 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 11:57:59.541813 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 11:57:59.541813 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.creator() May 19 11:57:59.541813 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 11:57:59.541813 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application created_value = creator( May 19 11:57:59.542653 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 11:57:59.542653 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 11:57:59.542653 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 11:57:59.542653 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application May 19 11:57:59.543306 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 234/472] 10.222.0.23 () {56 vars in 1108 bytes} [Thu May 19 11:57:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 11:57:59.544787 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ee0e46-25c9-484d-b1c8-2ff50b6b587b None None] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:59.545070 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ee0e46-25c9-484d-b1c8-2ff50b6b587b None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:59.546323 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ee0e46-25c9-484d-b1c8-2ff50b6b587b None None] PATH_INFO: `/` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:59.548469 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 235/473] 10.222.0.23 () {54 vars in 926 bytes} [Thu May 19 11:57:59 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 11:57:59.549768 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ee0e46-25c9-484d-b1c8-2ff50b6b587b None None] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:57:59.559102 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:57:59.582189 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8186f492-4611-46c0-af2c-6b736d582ed1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:57:59.583137 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8186f492-4611-46c0-af2c-6b736d582ed1 None admin] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:59.583499 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8186f492-4611-46c0-af2c-6b736d582ed1 None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:59.584023 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8186f492-4611-46c0-af2c-6b736d582ed1 None admin] PATH_INFO: `/v3/users` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:59.584814 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8186f492-4611-46c0-af2c-6b736d582ed1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:57:59.586524 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8186f492-4611-46c0-af2c-6b736d582ed1 None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:57:59.684286 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ee0e46-25c9-484d-b1c8-2ff50b6b587b None None] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:57:59.697951 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:57:59.710016 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 239/474] 10.222.0.23 () {60 vars in 1132 bytes} [Thu May 19 11:57:59 2022] POST /identity/v3/users => generated 248 bytes in 342 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 11:57:59.710016 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6716b8ba-11c2-4e22-8a60-032e1c143b9b None None] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:59.710016 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6716b8ba-11c2-4e22-8a60-032e1c143b9b None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:59.710945 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6716b8ba-11c2-4e22-8a60-032e1c143b9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:59.730863 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b4ac62-fe8e-43bf-96ff-d073e5436fdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:57:59.731293 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b4ac62-fe8e-43bf-96ff-d073e5436fdd None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:59.734176 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b4ac62-fe8e-43bf-96ff-d073e5436fdd None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:59.734176 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b4ac62-fe8e-43bf-96ff-d073e5436fdd None admin] PATH_INFO: `/v3/domains` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:59.734176 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b4ac62-fe8e-43bf-96ff-d073e5436fdd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 11:57:59.734176 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b4ac62-fe8e-43bf-96ff-d073e5436fdd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:57:59.735551 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b4ac62-fe8e-43bf-96ff-d073e5436fdd None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:57:59.739006 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.auth.core [None req-6716b8ba-11c2-4e22-8a60-032e1c143b9b None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 11:57:59.740369 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-6716b8ba-11c2-4e22-8a60-032e1c143b9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 11:57:59.749746 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 236/475] 10.222.0.23 () {56 vars in 1109 bytes} [Thu May 19 11:57:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:57:59.755134 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b4ac62-fe8e-43bf-96ff-d073e5436fdd None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:57:59.846449 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 240/476] 10.222.0.23 () {58 vars in 1023 bytes} [Thu May 19 11:57:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 11:57:59.872167 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6716b8ba-11c2-4e22-8a60-032e1c143b9b None None] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:57:59.927634 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b4ac62-fe8e-43bf-96ff-d073e5436fdd None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:57:59.984673 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee4aa9f-c6c9-479a-96cf-aa95b3abc6bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:57:59.985683 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee4aa9f-c6c9-479a-96cf-aa95b3abc6bd None admin] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:57:59.986138 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee4aa9f-c6c9-479a-96cf-aa95b3abc6bd None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:57:59.986546 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee4aa9f-c6c9-479a-96cf-aa95b3abc6bd None admin] PATH_INFO: `/v3/users` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:57:59.987485 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee4aa9f-c6c9-479a-96cf-aa95b3abc6bd None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:57:59.989223 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee4aa9f-c6c9-479a-96cf-aa95b3abc6bd None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:00.014025 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6716b8ba-11c2-4e22-8a60-032e1c143b9b None None] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:00.025087 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:58:00.054871 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa843a4d-0193-41d7-a767-4857466cf20b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:00.055689 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa843a4d-0193-41d7-a767-4857466cf20b None admin] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:00.056101 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa843a4d-0193-41d7-a767-4857466cf20b None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:00.056539 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa843a4d-0193-41d7-a767-4857466cf20b None admin] PATH_INFO: `/v3/roles` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:00.057428 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa843a4d-0193-41d7-a767-4857466cf20b None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:00.058901 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa843a4d-0193-41d7-a767-4857466cf20b None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:00.090539 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.sql.core [None req-aa843a4d-0193-41d7-a767-4857466cf20b None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") May 19 11:58:00.090539 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] May 19 11:58:00.090539 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [parameters: {'id': '116fc1859ee44a6fa1eb74ae92a1ce22', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] May 19 11:58:00.090539 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80540) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} May 19 11:58:00.104721 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 237/477] 10.222.0.23 () {60 vars in 1132 bytes} [Thu May 19 11:57:59 2022] POST /identity/v3/users => generated 249 bytes in 349 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 11:58:00.108719 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application [None req-aa843a4d-0193-41d7-a767-4857466cf20b None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. May 19 11:58:00.108719 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 11:58:00.108719 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 19 11:58:00.108719 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application self.dialect.do_execute( May 19 11:58:00.108719 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 19 11:58:00.108719 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 19 11:58:00.108719 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 19 11:58:00.108719 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application result = self._query(query) May 19 11:58:00.108719 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 19 11:58:00.108719 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application conn.query(q) May 19 11:58:00.108719 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 19 11:58:00.108719 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 19 11:58:00.108719 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 19 11:58:00.108719 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application result.read() May 19 11:58:00.108719 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 19 11:58:00.109600 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 19 11:58:00.109600 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 19 11:58:00.109600 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application packet.raise_for_error() May 19 11:58:00.109600 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 19 11:58:00.109600 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 19 11:58:00.109600 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 19 11:58:00.109600 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 19 11:58:00.109600 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") May 19 11:58:00.109600 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application May 19 11:58:00.109600 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: May 19 11:58:00.109600 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application May 19 11:58:00.109600 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 11:58:00.109600 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper May 19 11:58:00.109600 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return method(*args, **kwargs) May 19 11:58:00.109600 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role May 19 11:58:00.109600 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return ref.to_dict() May 19 11:58:00.109600 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 19 11:58:00.109600 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application next(self.gen) May 19 11:58:00.110479 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope May 19 11:58:00.110479 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application yield resource May 19 11:58:00.110479 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 19 11:58:00.110479 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application next(self.gen) May 19 11:58:00.110479 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session May 19 11:58:00.110479 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application self.session.rollback() May 19 11:58:00.110479 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ May 19 11:58:00.110479 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application self.force_reraise() May 19 11:58:00.110479 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise May 19 11:58:00.110479 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application raise self.value May 19 11:58:00.110479 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session May 19 11:58:00.110479 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) May 19 11:58:00.110479 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction May 19 11:58:00.110479 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application session.commit() May 19 11:58:00.110479 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1435, in commit May 19 11:58:00.110479 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) May 19 11:58:00.110479 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit May 19 11:58:00.111528 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application self._prepare_impl() May 19 11:58:00.111528 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl May 19 11:58:00.111528 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application self.session.flush() May 19 11:58:00.111528 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3367, in flush May 19 11:58:00.111528 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application self._flush(objects) May 19 11:58:00.111528 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3507, in _flush May 19 11:58:00.111528 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) May 19 11:58:00.111528 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ May 19 11:58:00.111528 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application compat.raise_( May 19 11:58:00.111528 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 19 11:58:00.111528 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application raise exception May 19 11:58:00.111528 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3467, in _flush May 19 11:58:00.111528 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application flush_context.execute() May 19 11:58:00.111528 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute May 19 11:58:00.111528 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application rec.execute(self) May 19 11:58:00.111528 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute May 19 11:58:00.111528 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( May 19 11:58:00.112478 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj May 19 11:58:00.112478 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application _emit_insert_statements( May 19 11:58:00.112478 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements May 19 11:58:00.112478 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application c = connection._execute_20( May 19 11:58:00.112478 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 May 19 11:58:00.112478 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) May 19 11:58:00.112478 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection May 19 11:58:00.112478 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return connection._execute_clauseelement( May 19 11:58:00.112478 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement May 19 11:58:00.112478 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application ret = self._execute_context( May 19 11:58:00.112478 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context May 19 11:58:00.112478 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application self._handle_dbapi_exception( May 19 11:58:00.112478 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception May 19 11:58:00.112478 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) May 19 11:58:00.112478 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 19 11:58:00.112478 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application raise exception May 19 11:58:00.113346 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 19 11:58:00.113346 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application self.dialect.do_execute( May 19 11:58:00.113346 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 19 11:58:00.113346 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 19 11:58:00.113346 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 19 11:58:00.113346 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application result = self._query(query) May 19 11:58:00.113346 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 19 11:58:00.113346 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application conn.query(q) May 19 11:58:00.113346 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 19 11:58:00.113346 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 19 11:58:00.113346 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 19 11:58:00.113346 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application result.read() May 19 11:58:00.113346 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 19 11:58:00.113346 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 19 11:58:00.113346 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 19 11:58:00.113346 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application packet.raise_for_error() May 19 11:58:00.114320 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 19 11:58:00.114320 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 19 11:58:00.114320 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 19 11:58:00.114320 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 19 11:58:00.114320 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") May 19 11:58:00.114320 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] May 19 11:58:00.114320 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application [parameters: {'id': '116fc1859ee44a6fa1eb74ae92a1ce22', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] May 19 11:58:00.114320 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) May 19 11:58:00.114320 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application May 19 11:58:00.114320 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 11:58:00.114320 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application May 19 11:58:00.114320 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 11:58:00.114320 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 11:58:00.114320 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 11:58:00.114320 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 11:58:00.114320 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 11:58:00.115315 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 11:58:00.115315 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 11:58:00.115315 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 11:58:00.115315 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 11:58:00.115315 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 11:58:00.115315 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 11:58:00.115315 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post May 19 11:58:00.115315 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( May 19 11:58:00.115315 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 11:58:00.115315 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 11:58:00.115315 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role May 19 11:58:00.115315 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) May 19 11:58:00.115315 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper May 19 11:58:00.115315 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, May 19 11:58:00.115315 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. May 19 11:58:00.115315 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application May 19 11:58:00.116331 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 241/478] 10.222.0.23 () {60 vars in 1131 bytes} [Thu May 19 11:57:59 2022] POST /identity/v3/roles => generated 144 bytes in 239 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) May 19 11:58:00.116407 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee4aa9f-c6c9-479a-96cf-aa95b3abc6bd None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:00.151693 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:58:00.269080 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee4aa9f-c6c9-479a-96cf-aa95b3abc6bd None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:00.333922 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605b9241-146b-4f13-bc24-fda340271ee8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:00.334777 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605b9241-146b-4f13-bc24-fda340271ee8 None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:00.335056 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605b9241-146b-4f13-bc24-fda340271ee8 None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:00.335325 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605b9241-146b-4f13-bc24-fda340271ee8 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:00.351349 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605b9241-146b-4f13-bc24-fda340271ee8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 11:58:00.360787 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application [None req-605b9241-146b-4f13-bc24-fda340271ee8 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 11:58:00.360787 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 11:58:00.360787 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 11:58:00.360787 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 11:58:00.360787 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 11:58:00.360787 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 11:58:00.360787 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 11:58:00.360787 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 11:58:00.360787 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 11:58:00.360787 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 11:58:00.360787 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 11:58:00.360787 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 11:58:00.360787 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 11:58:00.360787 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 11:58:00.360787 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 11:58:00.360787 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 11:58:00.362919 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 11:58:00.362919 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 11:58:00.362919 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 11:58:00.362919 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 11:58:00.362919 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 11:58:00.362919 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 11:58:00.362919 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 11:58:00.362919 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 11:58:00.362919 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 11:58:00.362919 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.get_or_create( May 19 11:58:00.362919 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 11:58:00.362919 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application with Lock( May 19 11:58:00.362919 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 11:58:00.362919 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._enter() May 19 11:58:00.362919 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 11:58:00.362919 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 11:58:00.364983 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 11:58:00.364983 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.creator() May 19 11:58:00.364983 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 11:58:00.364983 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application created_value = creator( May 19 11:58:00.364983 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 11:58:00.364983 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 11:58:00.364983 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 11:58:00.364983 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 11:58:00.364983 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 11:58:00.364983 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 11:58:00.364983 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 11:58:00.364983 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application May 19 11:58:00.368713 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 238/479] 10.222.0.23 () {56 vars in 1114 bytes} [Thu May 19 11:58:00 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 11:58:00.368903 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa843a4d-0193-41d7-a767-4857466cf20b None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:00.496124 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa843a4d-0193-41d7-a767-4857466cf20b None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:00.539892 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ba173b-dc63-44aa-bcec-fd743d6f65bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:00.540562 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ba173b-dc63-44aa-bcec-fd743d6f65bf None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:00.540844 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ba173b-dc63-44aa-bcec-fd743d6f65bf None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:00.541120 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ba173b-dc63-44aa-bcec-fd743d6f65bf None admin] PATH_INFO: `/v3/roles` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:00.541869 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ba173b-dc63-44aa-bcec-fd743d6f65bf None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 11:58:00.542232 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ba173b-dc63-44aa-bcec-fd743d6f65bf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:00.543628 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ba173b-dc63-44aa-bcec-fd743d6f65bf None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:00.552845 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 242/480] 10.222.0.23 () {56 vars in 1111 bytes} [Thu May 19 11:58:00 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:58:01.799126 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63b643f-90ce-4fb4-a047-06722735ae4f None None] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:01.799654 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63b643f-90ce-4fb4-a047-06722735ae4f None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:01.799764 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63b643f-90ce-4fb4-a047-06722735ae4f None None] PATH_INFO: `/` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:01.801339 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 239/481] 10.222.0.23 () {54 vars in 926 bytes} [Thu May 19 11:58:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 11:58:01.806788 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4058ba8a-20f7-4b7c-9443-3d32c98e355c None None] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:01.807248 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4058ba8a-20f7-4b7c-9443-3d32c98e355c None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:01.807337 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4058ba8a-20f7-4b7c-9443-3d32c98e355c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:01.825004 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.auth.core [None req-4058ba8a-20f7-4b7c-9443-3d32c98e355c None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 11:58:01.826205 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-4058ba8a-20f7-4b7c-9443-3d32c98e355c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 11:58:01.858980 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:58:01.931570 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 243/482] 10.222.0.23 () {58 vars in 1023 bytes} [Thu May 19 11:58:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 11:58:01.956600 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63b643f-90ce-4fb4-a047-06722735ae4f None None] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:02.055917 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:58:02.080092 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63b643f-90ce-4fb4-a047-06722735ae4f None None] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:02.130250 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f73372-4a6d-4da8-be0a-6d2c4e6d0506 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:02.131352 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f73372-4a6d-4da8-be0a-6d2c4e6d0506 None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:02.131806 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f73372-4a6d-4da8-be0a-6d2c4e6d0506 None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:02.132230 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f73372-4a6d-4da8-be0a-6d2c4e6d0506 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:02.152723 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f73372-4a6d-4da8-be0a-6d2c4e6d0506 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 11:58:02.161177 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d7f353-5e73-478b-9a63-71d8b906de20 None None] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:02.161726 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d7f353-5e73-478b-9a63-71d8b906de20 None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:02.162151 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d7f353-5e73-478b-9a63-71d8b906de20 None None] PATH_INFO: `/` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:02.163678 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 244/483] 10.222.0.23 () {54 vars in 926 bytes} [Thu May 19 11:58:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 11:58:02.166619 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application [None req-94f73372-4a6d-4da8-be0a-6d2c4e6d0506 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 11:58:02.166619 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 11:58:02.166619 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 11:58:02.166619 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 11:58:02.166619 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 11:58:02.166619 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 11:58:02.166619 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 11:58:02.166619 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 11:58:02.166619 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 11:58:02.166619 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 11:58:02.166619 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 11:58:02.166619 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 11:58:02.166619 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 11:58:02.166619 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 11:58:02.166619 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 11:58:02.166619 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 11:58:02.167699 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 11:58:02.167699 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 11:58:02.167699 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 11:58:02.167699 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 11:58:02.167699 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 11:58:02.167699 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 11:58:02.167699 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 11:58:02.167699 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 11:58:02.167699 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 11:58:02.167699 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.get_or_create( May 19 11:58:02.167699 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 11:58:02.167699 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application with Lock( May 19 11:58:02.167699 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 11:58:02.167699 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._enter() May 19 11:58:02.167699 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 11:58:02.167699 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 11:58:02.168697 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 11:58:02.168697 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.creator() May 19 11:58:02.168697 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 11:58:02.168697 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application created_value = creator( May 19 11:58:02.168697 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 11:58:02.168697 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 11:58:02.168697 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 11:58:02.168697 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 11:58:02.168697 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 11:58:02.168697 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 11:58:02.168697 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 11:58:02.168697 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application May 19 11:58:02.172215 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c371d4f-1cfb-4050-9f5d-51d173b59ab8 None None] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:02.172694 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c371d4f-1cfb-4050-9f5d-51d173b59ab8 None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:02.173116 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c371d4f-1cfb-4050-9f5d-51d173b59ab8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:02.188990 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 240/484] 10.222.0.23 () {56 vars in 1102 bytes} [Thu May 19 11:58:01 2022] GET /identity/v3/roles/service => generated 85 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 11:58:02.190838 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f73372-4a6d-4da8-be0a-6d2c4e6d0506 None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:02.205877 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.auth.core [None req-2c371d4f-1cfb-4050-9f5d-51d173b59ab8 None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 11:58:02.207339 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-2c371d4f-1cfb-4050-9f5d-51d173b59ab8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 11:58:02.364487 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 245/485] 10.222.0.23 () {58 vars in 1023 bytes} [Thu May 19 11:58:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 11:58:02.385827 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f73372-4a6d-4da8-be0a-6d2c4e6d0506 None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:02.410764 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c371d4f-1cfb-4050-9f5d-51d173b59ab8 None None] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:02.438278 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d2b656-068a-4336-8432-08e4ac4c4e66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:02.438965 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d2b656-068a-4336-8432-08e4ac4c4e66 None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:02.439247 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d2b656-068a-4336-8432-08e4ac4c4e66 None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:02.439518 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d2b656-068a-4336-8432-08e4ac4c4e66 None admin] PATH_INFO: `/v3/roles` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:02.441450 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d2b656-068a-4336-8432-08e4ac4c4e66 None admin] RBAC: Adding query filter params (name=service) {{(pid=80539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 11:58:02.441759 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d2b656-068a-4336-8432-08e4ac4c4e66 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:02.443208 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d2b656-068a-4336-8432-08e4ac4c4e66 None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:02.462653 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 241/486] 10.222.0.23 () {56 vars in 1103 bytes} [Thu May 19 11:58:02 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 11:58:02.467862 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d2b656-068a-4336-8432-08e4ac4c4e66 None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:02.484941 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:58:02.547270 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:58:02.556797 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c371d4f-1cfb-4050-9f5d-51d173b59ab8 None None] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:02.596559 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59715c11-da48-40fe-b216-aa0d40b71cd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:02.597464 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59715c11-da48-40fe-b216-aa0d40b71cd2 None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:02.597819 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59715c11-da48-40fe-b216-aa0d40b71cd2 None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:02.598167 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59715c11-da48-40fe-b216-aa0d40b71cd2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:02.611590 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59715c11-da48-40fe-b216-aa0d40b71cd2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 11:58:02.614656 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d2b656-068a-4336-8432-08e4ac4c4e66 None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:02.620077 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application [None req-59715c11-da48-40fe-b216-aa0d40b71cd2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 11:58:02.620077 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 11:58:02.620077 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 11:58:02.620077 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 11:58:02.620077 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 11:58:02.620077 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 11:58:02.620077 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 11:58:02.620077 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 11:58:02.620077 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 11:58:02.620077 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 11:58:02.620077 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 11:58:02.620077 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 11:58:02.620077 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 11:58:02.620077 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 11:58:02.620077 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 11:58:02.620077 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 11:58:02.621029 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 11:58:02.621029 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 11:58:02.621029 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 11:58:02.621029 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 11:58:02.621029 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 11:58:02.621029 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 11:58:02.621029 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 11:58:02.621029 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 11:58:02.621029 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 11:58:02.621029 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.get_or_create( May 19 11:58:02.621029 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 11:58:02.621029 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application with Lock( May 19 11:58:02.621029 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 11:58:02.621029 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._enter() May 19 11:58:02.621029 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 11:58:02.621029 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 11:58:02.621800 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 11:58:02.621800 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.creator() May 19 11:58:02.621800 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 11:58:02.621800 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application created_value = creator( May 19 11:58:02.621800 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 11:58:02.621800 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 11:58:02.621800 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 11:58:02.621800 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 11:58:02.621800 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 11:58:02.621800 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 11:58:02.621800 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 11:58:02.621800 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application May 19 11:58:02.622394 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 246/487] 10.222.0.23 () {56 vars in 1102 bytes} [Thu May 19 11:58:02 2022] GET /identity/v3/roles/service => generated 85 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 11:58:02.623684 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f51b86-a08e-4994-b3d6-137139784623 None None] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:02.625142 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f51b86-a08e-4994-b3d6-137139784623 None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:02.625561 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f51b86-a08e-4994-b3d6-137139784623 None None] PATH_INFO: `/` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:02.627295 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 247/488] 10.222.0.23 () {54 vars in 926 bytes} [Thu May 19 11:58:02 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 11:58:02.628092 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f51b86-a08e-4994-b3d6-137139784623 None None] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:02.671388 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a4892f-22bd-4a39-9a66-65d08d851dbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:02.672894 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a4892f-22bd-4a39-9a66-65d08d851dbb None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:02.673578 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a4892f-22bd-4a39-9a66-65d08d851dbb None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:02.674003 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a4892f-22bd-4a39-9a66-65d08d851dbb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:02.689719 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a4892f-22bd-4a39-9a66-65d08d851dbb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:02.691991 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a4892f-22bd-4a39-9a66-65d08d851dbb None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:02.714902 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application [None req-c8a4892f-22bd-4a39-9a66-65d08d851dbb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 11:58:02.714902 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 11:58:02.714902 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 11:58:02.714902 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 11:58:02.714902 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 11:58:02.714902 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 11:58:02.714902 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 11:58:02.714902 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 11:58:02.714902 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 11:58:02.714902 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 11:58:02.714902 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 11:58:02.714902 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 11:58:02.714902 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 11:58:02.714902 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 11:58:02.714902 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 11:58:02.716098 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 11:58:02.716098 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 11:58:02.716098 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 11:58:02.716098 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 11:58:02.716098 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 11:58:02.716098 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 11:58:02.716098 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.get_or_create( May 19 11:58:02.716098 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 11:58:02.716098 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application with Lock( May 19 11:58:02.716098 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 11:58:02.716098 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._enter() May 19 11:58:02.716098 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 11:58:02.716098 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 11:58:02.716098 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 11:58:02.716098 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.creator() May 19 11:58:02.716098 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 11:58:02.716098 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application created_value = creator( May 19 11:58:02.717186 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 11:58:02.717186 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 11:58:02.717186 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 11:58:02.717186 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application May 19 11:58:02.723280 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 242/489] 10.222.0.23 () {56 vars in 1108 bytes} [Thu May 19 11:58:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 11:58:02.723280 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1bdc4c-6db3-46d3-b70a-1b935f62b85a None None] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:02.723280 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1bdc4c-6db3-46d3-b70a-1b935f62b85a None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:02.723280 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1bdc4c-6db3-46d3-b70a-1b935f62b85a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:02.742187 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.auth.core [None req-bc1bdc4c-6db3-46d3-b70a-1b935f62b85a None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 11:58:02.744368 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-bc1bdc4c-6db3-46d3-b70a-1b935f62b85a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 11:58:02.770634 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f51b86-a08e-4994-b3d6-137139784623 None None] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:02.811771 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9406467a-b035-4892-94fe-0a8831e54b90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:02.813423 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9406467a-b035-4892-94fe-0a8831e54b90 None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:02.814169 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9406467a-b035-4892-94fe-0a8831e54b90 None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:02.814788 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9406467a-b035-4892-94fe-0a8831e54b90 None admin] PATH_INFO: `/v3/roles` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:02.815872 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9406467a-b035-4892-94fe-0a8831e54b90 None admin] RBAC: Adding query filter params (name=service) {{(pid=80540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 11:58:02.816389 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9406467a-b035-4892-94fe-0a8831e54b90 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:02.818188 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9406467a-b035-4892-94fe-0a8831e54b90 None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:02.820768 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:58:02.829796 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 248/490] 10.222.0.23 () {56 vars in 1103 bytes} [Thu May 19 11:58:02 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:58:02.831053 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9406467a-b035-4892-94fe-0a8831e54b90 None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:02.835869 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 243/491] 10.222.0.23 () {58 vars in 1023 bytes} [Thu May 19 11:58:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 11:58:02.837038 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1bdc4c-6db3-46d3-b70a-1b935f62b85a None None] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:02.866088 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:58:02.972177 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9406467a-b035-4892-94fe-0a8831e54b90 None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:02.994364 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1bdc4c-6db3-46d3-b70a-1b935f62b85a None None] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:03.015436 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd209a6f-faa9-45ee-908d-9f9d50d71eb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:03.016635 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd209a6f-faa9-45ee-908d-9f9d50d71eb8 None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:03.017119 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd209a6f-faa9-45ee-908d-9f9d50d71eb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:03.017605 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd209a6f-faa9-45ee-908d-9f9d50d71eb8 None admin] PATH_INFO: `/v3/domains` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:03.018552 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd209a6f-faa9-45ee-908d-9f9d50d71eb8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 11:58:03.019103 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd209a6f-faa9-45ee-908d-9f9d50d71eb8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:03.020944 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd209a6f-faa9-45ee-908d-9f9d50d71eb8 None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:03.040269 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 249/492] 10.222.0.23 () {56 vars in 1109 bytes} [Thu May 19 11:58:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:58:03.040269 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd209a6f-faa9-45ee-908d-9f9d50d71eb8 None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:03.044035 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c18f25-e518-4637-b060-1bcb42d917b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:03.046422 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c18f25-e518-4637-b060-1bcb42d917b8 None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:03.046832 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c18f25-e518-4637-b060-1bcb42d917b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:03.047046 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c18f25-e518-4637-b060-1bcb42d917b8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:03.060226 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c18f25-e518-4637-b060-1bcb42d917b8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:03.062579 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c18f25-e518-4637-b060-1bcb42d917b8 None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:03.083536 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application [None req-52c18f25-e518-4637-b060-1bcb42d917b8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 11:58:03.083536 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 11:58:03.083536 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 11:58:03.083536 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 11:58:03.083536 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 11:58:03.083536 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 11:58:03.083536 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 11:58:03.083536 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 11:58:03.083536 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 11:58:03.083536 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 11:58:03.083536 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 11:58:03.083536 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 11:58:03.083536 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 11:58:03.083536 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 11:58:03.083536 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 11:58:03.084526 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 11:58:03.084526 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 11:58:03.084526 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 11:58:03.084526 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 11:58:03.084526 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 11:58:03.084526 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 11:58:03.084526 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.get_or_create( May 19 11:58:03.084526 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 11:58:03.084526 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application with Lock( May 19 11:58:03.084526 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 11:58:03.084526 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._enter() May 19 11:58:03.084526 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 11:58:03.084526 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 11:58:03.084526 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 11:58:03.084526 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.creator() May 19 11:58:03.084526 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 11:58:03.084526 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application created_value = creator( May 19 11:58:03.086017 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 11:58:03.086017 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 11:58:03.086017 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 11:58:03.086017 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application May 19 11:58:03.086017 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 244/493] 10.222.0.23 () {56 vars in 1108 bytes} [Thu May 19 11:58:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 11:58:03.086017 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c18f25-e518-4637-b060-1bcb42d917b8 None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:03.147428 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:58:03.163662 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:58:03.200348 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd209a6f-faa9-45ee-908d-9f9d50d71eb8 None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:03.226049 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c18f25-e518-4637-b060-1bcb42d917b8 None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:03.245126 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07889f0-bc35-4bd4-96e3-60ec558166dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:03.246030 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07889f0-bc35-4bd4-96e3-60ec558166dc None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:03.246320 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07889f0-bc35-4bd4-96e3-60ec558166dc None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:03.246654 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07889f0-bc35-4bd4-96e3-60ec558166dc None admin] PATH_INFO: `/v3/roles/service` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:03.260048 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07889f0-bc35-4bd4-96e3-60ec558166dc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 11:58:03.268707 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application [None req-b07889f0-bc35-4bd4-96e3-60ec558166dc None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 11:58:03.268707 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 11:58:03.268707 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 11:58:03.268707 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 11:58:03.268707 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 11:58:03.268707 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 11:58:03.268707 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 11:58:03.268707 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 11:58:03.268707 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 11:58:03.268707 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 11:58:03.268707 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 11:58:03.268707 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 11:58:03.268707 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 11:58:03.268707 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 11:58:03.268707 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 11:58:03.268707 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 11:58:03.269708 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 11:58:03.269708 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 11:58:03.269708 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 11:58:03.269708 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 11:58:03.269708 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 11:58:03.269708 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 11:58:03.269708 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 11:58:03.269708 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 11:58:03.269708 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 11:58:03.269708 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.get_or_create( May 19 11:58:03.269708 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 11:58:03.269708 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application with Lock( May 19 11:58:03.269708 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 11:58:03.269708 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._enter() May 19 11:58:03.269708 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 11:58:03.269708 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 11:58:03.270525 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 11:58:03.270525 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.creator() May 19 11:58:03.270525 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 11:58:03.270525 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application created_value = creator( May 19 11:58:03.270525 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 11:58:03.270525 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 11:58:03.270525 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 11:58:03.270525 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 11:58:03.270525 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 11:58:03.270525 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 11:58:03.270525 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 11:58:03.270525 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application May 19 11:58:03.271836 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f640e8da-c9a5-4479-a8c2-71ed1c9c4e4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:03.273069 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f640e8da-c9a5-4479-a8c2-71ed1c9c4e4b None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:03.273561 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f640e8da-c9a5-4479-a8c2-71ed1c9c4e4b None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:03.274038 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f640e8da-c9a5-4479-a8c2-71ed1c9c4e4b None admin] PATH_INFO: `/v3/users/nova` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:03.274809 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 250/494] 10.222.0.23 () {56 vars in 1102 bytes} [Thu May 19 11:58:03 2022] GET /identity/v3/roles/service => generated 85 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 11:58:03.275713 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07889f0-bc35-4bd4-96e3-60ec558166dc None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:03.283731 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f640e8da-c9a5-4479-a8c2-71ed1c9c4e4b None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:03.285570 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f640e8da-c9a5-4479-a8c2-71ed1c9c4e4b None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:03.303084 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application [None req-f640e8da-c9a5-4479-a8c2-71ed1c9c4e4b None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 19 11:58:03.303084 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 11:58:03.303084 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 11:58:03.303084 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 11:58:03.303084 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 11:58:03.303084 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 11:58:03.303084 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 11:58:03.303084 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 11:58:03.303084 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 11:58:03.303084 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 11:58:03.303084 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 11:58:03.303084 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 11:58:03.303084 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 11:58:03.303084 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 11:58:03.303084 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 11:58:03.303084 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 11:58:03.304086 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 11:58:03.304086 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 11:58:03.304086 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 11:58:03.304086 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 11:58:03.304086 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 11:58:03.304086 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 11:58:03.304086 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 11:58:03.304086 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 11:58:03.304086 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 11:58:03.304086 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.get_or_create( May 19 11:58:03.304086 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 11:58:03.304086 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application with Lock( May 19 11:58:03.304086 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 11:58:03.304086 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._enter() May 19 11:58:03.304086 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 11:58:03.304086 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 11:58:03.304086 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 11:58:03.304086 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.creator() May 19 11:58:03.304984 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 11:58:03.304984 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application created_value = creator( May 19 11:58:03.304984 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 11:58:03.304984 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 11:58:03.304984 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 11:58:03.304984 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 11:58:03.304984 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 11:58:03.304984 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 11:58:03.304984 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 19 11:58:03.304984 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application May 19 11:58:03.308466 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 245/495] 10.222.0.23 () {56 vars in 1093 bytes} [Thu May 19 11:58:03 2022] GET /identity/v3/users/nova => generated 82 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 11:58:03.309638 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f640e8da-c9a5-4479-a8c2-71ed1c9c4e4b None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:03.390652 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07889f0-bc35-4bd4-96e3-60ec558166dc None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:03.417302 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f640e8da-c9a5-4479-a8c2-71ed1c9c4e4b None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:03.418790 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:58:03.427672 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ea3511-7e69-408d-9032-683794a39ee8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:03.429154 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ea3511-7e69-408d-9032-683794a39ee8 None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:03.429154 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ea3511-7e69-408d-9032-683794a39ee8 None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:03.429154 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ea3511-7e69-408d-9032-683794a39ee8 None admin] PATH_INFO: `/v3/domains` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:03.429674 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ea3511-7e69-408d-9032-683794a39ee8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 11:58:03.429935 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ea3511-7e69-408d-9032-683794a39ee8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:03.431396 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ea3511-7e69-408d-9032-683794a39ee8 None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:03.433088 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:58:03.447530 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 251/496] 10.222.0.23 () {56 vars in 1109 bytes} [Thu May 19 11:58:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:58:03.448455 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ea3511-7e69-408d-9032-683794a39ee8 None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:03.465303 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca615c6e-43f5-4a02-80f7-551aab48d019 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:03.465999 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca615c6e-43f5-4a02-80f7-551aab48d019 None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:03.466286 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca615c6e-43f5-4a02-80f7-551aab48d019 None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:03.466562 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca615c6e-43f5-4a02-80f7-551aab48d019 None admin] PATH_INFO: `/v3/roles` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:03.467333 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca615c6e-43f5-4a02-80f7-551aab48d019 None admin] RBAC: Adding query filter params (name=service) {{(pid=80539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 11:58:03.467642 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca615c6e-43f5-4a02-80f7-551aab48d019 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:03.469171 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca615c6e-43f5-4a02-80f7-551aab48d019 None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:03.478726 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 246/497] 10.222.0.23 () {56 vars in 1103 bytes} [Thu May 19 11:58:03 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:58:03.480145 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaea6f47-bf92-4738-a09f-95710e2e4784 None None] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:03.480435 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaea6f47-bf92-4738-a09f-95710e2e4784 None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:03.480713 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaea6f47-bf92-4738-a09f-95710e2e4784 None None] PATH_INFO: `/` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:03.482247 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 247/498] 10.222.0.23 () {54 vars in 926 bytes} [Thu May 19 11:58:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 11:58:03.486989 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaea6f47-bf92-4738-a09f-95710e2e4784 None None] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:03.583010 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ea3511-7e69-408d-9032-683794a39ee8 None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:03.610618 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaea6f47-bf92-4738-a09f-95710e2e4784 None None] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:03.624885 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7359ffc-a21c-44ec-870d-90f992b8e14a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:03.625562 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7359ffc-a21c-44ec-870d-90f992b8e14a None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:03.625847 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7359ffc-a21c-44ec-870d-90f992b8e14a None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:03.626120 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7359ffc-a21c-44ec-870d-90f992b8e14a None admin] PATH_INFO: `/v3/users` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:03.626963 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7359ffc-a21c-44ec-870d-90f992b8e14a None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 11:58:03.627277 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7359ffc-a21c-44ec-870d-90f992b8e14a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:03.628966 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7359ffc-a21c-44ec-870d-90f992b8e14a None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:03.649687 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d52af41-5260-4099-8b49-e1df09d76e5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:03.650518 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d52af41-5260-4099-8b49-e1df09d76e5d None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:03.650518 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d52af41-5260-4099-8b49-e1df09d76e5d None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:03.650518 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d52af41-5260-4099-8b49-e1df09d76e5d None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:03.659769 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d52af41-5260-4099-8b49-e1df09d76e5d None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:03.661251 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d52af41-5260-4099-8b49-e1df09d76e5d None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:03.668685 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 252/499] 10.222.0.23 () {56 vars in 1133 bytes} [Thu May 19 11:58:03 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:58:03.669456 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7359ffc-a21c-44ec-870d-90f992b8e14a None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:03.676398 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application [None req-0d52af41-5260-4099-8b49-e1df09d76e5d None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 19 11:58:03.676398 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 11:58:03.676398 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 11:58:03.676398 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 11:58:03.676398 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 11:58:03.676398 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 11:58:03.676398 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 11:58:03.676398 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 11:58:03.676398 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 11:58:03.676398 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 11:58:03.676398 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 11:58:03.676398 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 11:58:03.676398 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 11:58:03.676398 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 11:58:03.676398 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 11:58:03.676398 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 11:58:03.677551 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 11:58:03.677551 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 11:58:03.677551 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 11:58:03.677551 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 11:58:03.677551 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 11:58:03.677551 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 11:58:03.677551 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 11:58:03.677551 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 11:58:03.677551 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 11:58:03.677551 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.get_or_create( May 19 11:58:03.677551 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 11:58:03.677551 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application with Lock( May 19 11:58:03.677551 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 11:58:03.677551 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._enter() May 19 11:58:03.677551 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 11:58:03.677551 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 11:58:03.677551 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 11:58:03.677551 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.creator() May 19 11:58:03.678631 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 11:58:03.678631 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application created_value = creator( May 19 11:58:03.678631 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 11:58:03.678631 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 11:58:03.678631 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 11:58:03.678631 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 11:58:03.678631 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 11:58:03.678631 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 11:58:03.678631 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 19 11:58:03.678631 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application May 19 11:58:03.678631 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 248/500] 10.222.0.23 () {56 vars in 1099 bytes} [Thu May 19 11:58:03 2022] GET /identity/v3/users/cinder => generated 84 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 11:58:03.679322 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378e30dd-18ef-434b-b92b-41c5757d99c7 None None] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:03.679411 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378e30dd-18ef-434b-b92b-41c5757d99c7 None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:03.679644 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378e30dd-18ef-434b-b92b-41c5757d99c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:03.697514 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.auth.core [None req-378e30dd-18ef-434b-b92b-41c5757d99c7 None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 11:58:03.698722 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-378e30dd-18ef-434b-b92b-41c5757d99c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 11:58:03.716878 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:58:03.726960 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:58:03.767749 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 249/501] 10.222.0.23 () {58 vars in 1023 bytes} [Thu May 19 11:58:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 11:58:03.769126 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f696290-f556-4693-91aa-ba940af7e0bc None None] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:03.769417 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f696290-f556-4693-91aa-ba940af7e0bc None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:03.769692 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f696290-f556-4693-91aa-ba940af7e0bc None None] PATH_INFO: `/` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:03.770966 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 250/502] 10.222.0.23 () {54 vars in 926 bytes} [Thu May 19 11:58:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 11:58:03.772049 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f696290-f556-4693-91aa-ba940af7e0bc None None] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:03.777582 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7359ffc-a21c-44ec-870d-90f992b8e14a None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:03.813472 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1980809-921c-46f5-b09e-35bd65a50889 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:03.814277 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1980809-921c-46f5-b09e-35bd65a50889 None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:03.814568 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1980809-921c-46f5-b09e-35bd65a50889 None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:03.814847 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1980809-921c-46f5-b09e-35bd65a50889 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:03.826363 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1980809-921c-46f5-b09e-35bd65a50889 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:03.828547 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1980809-921c-46f5-b09e-35bd65a50889 None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:03.846488 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application [None req-e1980809-921c-46f5-b09e-35bd65a50889 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 11:58:03.846488 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 11:58:03.846488 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 11:58:03.846488 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 11:58:03.846488 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 11:58:03.846488 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 11:58:03.846488 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 11:58:03.846488 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 11:58:03.846488 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 11:58:03.846488 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 11:58:03.846488 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 11:58:03.846488 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 11:58:03.846488 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 11:58:03.846488 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 11:58:03.846488 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 11:58:03.847604 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 11:58:03.847604 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 11:58:03.847604 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 11:58:03.847604 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 11:58:03.847604 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 11:58:03.847604 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 11:58:03.847604 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.get_or_create( May 19 11:58:03.847604 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 11:58:03.847604 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application with Lock( May 19 11:58:03.847604 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 11:58:03.847604 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._enter() May 19 11:58:03.847604 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 11:58:03.847604 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 11:58:03.847604 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 11:58:03.847604 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.creator() May 19 11:58:03.847604 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 11:58:03.847604 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application created_value = creator( May 19 11:58:03.848643 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 11:58:03.848643 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 11:58:03.848643 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 11:58:03.848643 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application May 19 11:58:03.848643 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 253/503] 10.222.0.23 () {56 vars in 1108 bytes} [Thu May 19 11:58:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 11:58:03.848643 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1980809-921c-46f5-b09e-35bd65a50889 None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:03.895887 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f696290-f556-4693-91aa-ba940af7e0bc None None] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:03.926860 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ec5e59-918b-4454-a294-c7f47bc43c07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:03.927431 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ec5e59-918b-4454-a294-c7f47bc43c07 None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:03.927712 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ec5e59-918b-4454-a294-c7f47bc43c07 None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:03.928015 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ec5e59-918b-4454-a294-c7f47bc43c07 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:03.955897 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:58:03.959122 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1980809-921c-46f5-b09e-35bd65a50889 None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:03.961558 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:58:03.994503 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6c4c4e-8708-4030-9a58-efa71dff70f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:03.995450 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6c4c4e-8708-4030-9a58-efa71dff70f5 None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:03.995738 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6c4c4e-8708-4030-9a58-efa71dff70f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:03.996050 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6c4c4e-8708-4030-9a58-efa71dff70f5 None admin] PATH_INFO: `/v3/users` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:03.996891 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6c4c4e-8708-4030-9a58-efa71dff70f5 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 11:58:03.997202 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6c4c4e-8708-4030-9a58-efa71dff70f5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:03.998839 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6c4c4e-8708-4030-9a58-efa71dff70f5 None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:04.007995 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ec5e59-918b-4454-a294-c7f47bc43c07 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:04.009552 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ec5e59-918b-4454-a294-c7f47bc43c07 None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:04.017918 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 254/504] 10.222.0.23 () {56 vars in 1137 bytes} [Thu May 19 11:58:03 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:58:04.019265 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9de0ade-6bb1-4331-88c7-e067175be57b None None] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:04.019562 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9de0ade-6bb1-4331-88c7-e067175be57b None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:04.019997 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9de0ade-6bb1-4331-88c7-e067175be57b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:04.038826 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.auth.core [None req-b9de0ade-6bb1-4331-88c7-e067175be57b None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 11:58:04.040080 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-b9de0ade-6bb1-4331-88c7-e067175be57b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 11:58:04.110500 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 255/505] 10.222.0.23 () {58 vars in 1023 bytes} [Thu May 19 11:58:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 11:58:04.111370 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9de0ade-6bb1-4331-88c7-e067175be57b None None] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:04.116753 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 251/506] 10.222.0.23 () {58 vars in 1282 bytes} [Thu May 19 11:58:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 11:58:04.116753 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ec5e59-918b-4454-a294-c7f47bc43c07 None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:04.171484 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:58:04.209186 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:58:04.216631 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ec5e59-918b-4454-a294-c7f47bc43c07 None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:04.217505 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9de0ade-6bb1-4331-88c7-e067175be57b None None] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:04.251145 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d1a170-f49d-4a8b-891d-7b1b1de2b14c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:04.251835 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d1a170-f49d-4a8b-891d-7b1b1de2b14c None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:04.252123 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d1a170-f49d-4a8b-891d-7b1b1de2b14c None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:04.252397 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d1a170-f49d-4a8b-891d-7b1b1de2b14c None admin] PATH_INFO: `/v3/domains` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:04.252674 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68999638-0d29-466e-8a76-9ef82042562e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:04.253125 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d1a170-f49d-4a8b-891d-7b1b1de2b14c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 11:58:04.253431 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d1a170-f49d-4a8b-891d-7b1b1de2b14c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:04.253537 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68999638-0d29-466e-8a76-9ef82042562e None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:04.253756 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68999638-0d29-466e-8a76-9ef82042562e None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:04.254032 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68999638-0d29-466e-8a76-9ef82042562e None admin] PATH_INFO: `/v3/roles/service` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:04.254811 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d1a170-f49d-4a8b-891d-7b1b1de2b14c None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:04.264791 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68999638-0d29-466e-8a76-9ef82042562e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 11:58:04.266540 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 252/507] 10.222.0.23 () {56 vars in 1109 bytes} [Thu May 19 11:58:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:58:04.267360 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d1a170-f49d-4a8b-891d-7b1b1de2b14c None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:04.272625 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application [None req-68999638-0d29-466e-8a76-9ef82042562e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 11:58:04.272625 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 11:58:04.272625 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 11:58:04.272625 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 11:58:04.272625 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 11:58:04.272625 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 11:58:04.272625 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 11:58:04.272625 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 11:58:04.272625 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 11:58:04.272625 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 11:58:04.272625 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 11:58:04.272625 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 11:58:04.272625 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 11:58:04.272625 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 11:58:04.272625 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 11:58:04.272625 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 11:58:04.273746 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 11:58:04.273746 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 11:58:04.273746 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 11:58:04.273746 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 11:58:04.273746 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 11:58:04.273746 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 11:58:04.273746 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 11:58:04.273746 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 11:58:04.273746 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 11:58:04.273746 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.get_or_create( May 19 11:58:04.273746 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 11:58:04.273746 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application with Lock( May 19 11:58:04.273746 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 11:58:04.273746 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._enter() May 19 11:58:04.273746 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 11:58:04.273746 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 11:58:04.274596 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 11:58:04.274596 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.creator() May 19 11:58:04.274596 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 11:58:04.274596 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application created_value = creator( May 19 11:58:04.274596 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 11:58:04.274596 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 11:58:04.274596 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 11:58:04.274596 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 11:58:04.274596 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 11:58:04.274596 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 11:58:04.274596 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 11:58:04.274596 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application May 19 11:58:04.274596 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 256/508] 10.222.0.23 () {56 vars in 1102 bytes} [Thu May 19 11:58:04 2022] GET /identity/v3/roles/service => generated 85 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 11:58:04.275289 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68999638-0d29-466e-8a76-9ef82042562e None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:04.369805 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d1a170-f49d-4a8b-891d-7b1b1de2b14c None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:04.380209 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68999638-0d29-466e-8a76-9ef82042562e None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:04.384275 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:58:04.408704 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7984614-8369-4036-a5e3-2d5525ba82ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:04.426792 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7984614-8369-4036-a5e3-2d5525ba82ac None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:04.426792 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7984614-8369-4036-a5e3-2d5525ba82ac None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:04.426792 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7984614-8369-4036-a5e3-2d5525ba82ac None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:04.427063 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdc2098-393d-4202-a1c3-e26318d18240 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:04.427063 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdc2098-393d-4202-a1c3-e26318d18240 None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:04.427063 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdc2098-393d-4202-a1c3-e26318d18240 None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:04.427063 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdc2098-393d-4202-a1c3-e26318d18240 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:04.427539 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfdc2098-393d-4202-a1c3-e26318d18240 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:04.429082 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfdc2098-393d-4202-a1c3-e26318d18240 None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:04.447408 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application [None req-cfdc2098-393d-4202-a1c3-e26318d18240 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 11:58:04.447408 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 11:58:04.447408 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 11:58:04.447408 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 11:58:04.447408 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 11:58:04.447408 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 11:58:04.447408 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 11:58:04.447408 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 11:58:04.447408 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 11:58:04.447408 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 11:58:04.447408 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 11:58:04.447408 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 11:58:04.447408 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 11:58:04.447408 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 11:58:04.447408 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 11:58:04.448381 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 11:58:04.448381 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 11:58:04.448381 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 11:58:04.448381 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 11:58:04.448381 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 11:58:04.448381 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 11:58:04.448381 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.get_or_create( May 19 11:58:04.448381 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 11:58:04.448381 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application with Lock( May 19 11:58:04.448381 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 11:58:04.448381 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._enter() May 19 11:58:04.448381 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 11:58:04.448381 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 11:58:04.448381 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 11:58:04.448381 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.creator() May 19 11:58:04.448381 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 11:58:04.448381 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application created_value = creator( May 19 11:58:04.449218 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 11:58:04.449218 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 11:58:04.449218 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 11:58:04.449218 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application May 19 11:58:04.449218 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 257/509] 10.222.0.23 () {56 vars in 1108 bytes} [Thu May 19 11:58:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 11:58:04.449484 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdc2098-393d-4202-a1c3-e26318d18240 None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:04.454345 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:58:04.482233 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7984614-8369-4036-a5e3-2d5525ba82ac None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:04.484198 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7984614-8369-4036-a5e3-2d5525ba82ac None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:04.562413 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdc2098-393d-4202-a1c3-e26318d18240 None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:04.586873 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 253/510] 10.222.0.23 () {58 vars in 1282 bytes} [Thu May 19 11:58:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 11:58:04.587725 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7984614-8369-4036-a5e3-2d5525ba82ac None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:04.601169 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08ee4be-5ae1-4123-a434-e9353ad43084 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:04.601974 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08ee4be-5ae1-4123-a434-e9353ad43084 None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:04.602259 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08ee4be-5ae1-4123-a434-e9353ad43084 None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:04.602535 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08ee4be-5ae1-4123-a434-e9353ad43084 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:04.604458 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:58:04.614587 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08ee4be-5ae1-4123-a434-e9353ad43084 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:04.616172 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08ee4be-5ae1-4123-a434-e9353ad43084 None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:04.633792 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application [None req-e08ee4be-5ae1-4123-a434-e9353ad43084 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 11:58:04.633792 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 11:58:04.633792 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 11:58:04.633792 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 11:58:04.633792 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 11:58:04.633792 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 11:58:04.633792 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 11:58:04.633792 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 11:58:04.633792 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 11:58:04.633792 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 11:58:04.633792 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 11:58:04.633792 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 11:58:04.633792 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 11:58:04.633792 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 11:58:04.633792 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 11:58:04.634690 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 11:58:04.634690 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 11:58:04.634690 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 11:58:04.634690 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 11:58:04.634690 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 11:58:04.634690 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 11:58:04.634690 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.get_or_create( May 19 11:58:04.634690 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 11:58:04.634690 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application with Lock( May 19 11:58:04.634690 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 11:58:04.634690 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._enter() May 19 11:58:04.634690 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 11:58:04.634690 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 11:58:04.634690 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 11:58:04.634690 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.creator() May 19 11:58:04.634690 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 11:58:04.634690 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application created_value = creator( May 19 11:58:04.635529 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 11:58:04.635529 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 11:58:04.635529 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 11:58:04.635529 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application May 19 11:58:04.635529 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 258/511] 10.222.0.23 () {56 vars in 1108 bytes} [Thu May 19 11:58:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 11:58:04.635813 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08ee4be-5ae1-4123-a434-e9353ad43084 None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:04.696717 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:58:04.697827 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7984614-8369-4036-a5e3-2d5525ba82ac None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:04.737265 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dde1f08-cab4-488c-a038-e64f0a942640 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:04.737980 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dde1f08-cab4-488c-a038-e64f0a942640 None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:04.738261 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dde1f08-cab4-488c-a038-e64f0a942640 None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:04.738533 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dde1f08-cab4-488c-a038-e64f0a942640 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:04.743440 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08ee4be-5ae1-4123-a434-e9353ad43084 None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:04.745816 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dde1f08-cab4-488c-a038-e64f0a942640 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:04.747302 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dde1f08-cab4-488c-a038-e64f0a942640 None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:04.761850 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application [None req-9dde1f08-cab4-488c-a038-e64f0a942640 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. May 19 11:58:04.761850 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 11:58:04.761850 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 11:58:04.761850 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 11:58:04.761850 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 11:58:04.761850 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 11:58:04.761850 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 11:58:04.761850 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 11:58:04.761850 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 11:58:04.761850 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 11:58:04.761850 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 11:58:04.761850 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 11:58:04.761850 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 11:58:04.761850 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 11:58:04.761850 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 11:58:04.761850 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 11:58:04.762975 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 11:58:04.762975 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 11:58:04.762975 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 11:58:04.762975 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 11:58:04.762975 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 11:58:04.762975 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 11:58:04.762975 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 11:58:04.762975 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 11:58:04.762975 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 11:58:04.762975 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.get_or_create( May 19 11:58:04.762975 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 11:58:04.762975 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application with Lock( May 19 11:58:04.762975 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 11:58:04.762975 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._enter() May 19 11:58:04.762975 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 11:58:04.762975 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 11:58:04.762975 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 11:58:04.762975 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.creator() May 19 11:58:04.764270 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 11:58:04.764270 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application created_value = creator( May 19 11:58:04.764270 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 11:58:04.764270 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 11:58:04.764270 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 11:58:04.764270 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 11:58:04.764270 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 11:58:04.764270 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 11:58:04.764270 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. May 19 11:58:04.764270 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application May 19 11:58:04.767833 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 254/512] 10.222.0.23 () {56 vars in 1099 bytes} [Thu May 19 11:58:04 2022] GET /identity/v3/users/glance => generated 84 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 11:58:04.768720 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dde1f08-cab4-488c-a038-e64f0a942640 None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:04.778403 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc6a2ec-96dc-40b9-adca-da81537539c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:04.779088 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc6a2ec-96dc-40b9-adca-da81537539c6 None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:04.779382 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc6a2ec-96dc-40b9-adca-da81537539c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:04.780226 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc6a2ec-96dc-40b9-adca-da81537539c6 None admin] PATH_INFO: `/v3/roles` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:04.780985 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc6a2ec-96dc-40b9-adca-da81537539c6 None admin] RBAC: Adding query filter params (name=service) {{(pid=80540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 11:58:04.781285 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc6a2ec-96dc-40b9-adca-da81537539c6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:04.782708 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc6a2ec-96dc-40b9-adca-da81537539c6 None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:04.791248 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 259/513] 10.222.0.23 () {56 vars in 1103 bytes} [Thu May 19 11:58:04 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:58:04.792111 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc6a2ec-96dc-40b9-adca-da81537539c6 None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:04.844336 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:58:04.869307 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dde1f08-cab4-488c-a038-e64f0a942640 None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:04.893223 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc6a2ec-96dc-40b9-adca-da81537539c6 None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:04.904241 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a08969d-eb02-4b95-ac74-0cce51179f68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:04.905484 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a08969d-eb02-4b95-ac74-0cce51179f68 None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:04.905771 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a08969d-eb02-4b95-ac74-0cce51179f68 None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:04.906047 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a08969d-eb02-4b95-ac74-0cce51179f68 None admin] PATH_INFO: `/v3/domains` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:04.906779 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a08969d-eb02-4b95-ac74-0cce51179f68 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 11:58:04.907083 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a08969d-eb02-4b95-ac74-0cce51179f68 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:04.908500 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a08969d-eb02-4b95-ac74-0cce51179f68 None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:04.912484 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:58:04.924676 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 255/514] 10.222.0.23 () {56 vars in 1109 bytes} [Thu May 19 11:58:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:58:04.924676 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a08969d-eb02-4b95-ac74-0cce51179f68 None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:04.935475 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc68617f-f7a3-4c5c-ae2a-fa2d149a8521 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:04.936936 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc68617f-f7a3-4c5c-ae2a-fa2d149a8521 None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:04.937218 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc68617f-f7a3-4c5c-ae2a-fa2d149a8521 None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:04.937490 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc68617f-f7a3-4c5c-ae2a-fa2d149a8521 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:04.949608 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc68617f-f7a3-4c5c-ae2a-fa2d149a8521 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:04.951173 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc68617f-f7a3-4c5c-ae2a-fa2d149a8521 None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:04.969267 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application [None req-dc68617f-f7a3-4c5c-ae2a-fa2d149a8521 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 11:58:04.969267 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 11:58:04.969267 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 11:58:04.969267 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 11:58:04.969267 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 11:58:04.969267 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 11:58:04.969267 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 11:58:04.969267 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 11:58:04.969267 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 11:58:04.969267 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 11:58:04.969267 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 11:58:04.969267 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 11:58:04.969267 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 11:58:04.969267 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 11:58:04.969267 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 11:58:04.970178 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 11:58:04.970178 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 11:58:04.970178 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 11:58:04.970178 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 11:58:04.970178 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 11:58:04.970178 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 11:58:04.970178 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.get_or_create( May 19 11:58:04.970178 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 11:58:04.970178 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application with Lock( May 19 11:58:04.970178 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 11:58:04.970178 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._enter() May 19 11:58:04.970178 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 11:58:04.970178 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 11:58:04.970178 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 11:58:04.970178 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.creator() May 19 11:58:04.970178 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 11:58:04.970178 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application created_value = creator( May 19 11:58:04.971198 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 11:58:04.971198 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 11:58:04.971198 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 11:58:04.971198 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application May 19 11:58:04.971198 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 260/515] 10.222.0.23 () {56 vars in 1108 bytes} [Thu May 19 11:58:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 11:58:04.971889 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc68617f-f7a3-4c5c-ae2a-fa2d149a8521 None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:05.031827 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a08969d-eb02-4b95-ac74-0cce51179f68 None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:05.066249 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1af660-444c-45b0-ad32-f21a455c3b16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:05.066908 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1af660-444c-45b0-ad32-f21a455c3b16 None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:05.067189 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1af660-444c-45b0-ad32-f21a455c3b16 None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:05.067463 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1af660-444c-45b0-ad32-f21a455c3b16 None admin] PATH_INFO: `/v3/domains` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:05.068219 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1af660-444c-45b0-ad32-f21a455c3b16 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 11:58:05.068523 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1af660-444c-45b0-ad32-f21a455c3b16 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:05.070150 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1af660-444c-45b0-ad32-f21a455c3b16 None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:05.071814 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:58:05.075722 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc68617f-f7a3-4c5c-ae2a-fa2d149a8521 None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:05.083466 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 256/516] 10.222.0.23 () {56 vars in 1109 bytes} [Thu May 19 11:58:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:58:05.084347 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1af660-444c-45b0-ad32-f21a455c3b16 None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:05.117611 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3923c28-1b7e-48bc-9ba6-09af1a533545 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:05.118285 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3923c28-1b7e-48bc-9ba6-09af1a533545 None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:05.118572 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3923c28-1b7e-48bc-9ba6-09af1a533545 None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:05.118851 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3923c28-1b7e-48bc-9ba6-09af1a533545 None admin] PATH_INFO: `/v3/users` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:05.119692 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3923c28-1b7e-48bc-9ba6-09af1a533545 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 11:58:05.120035 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3923c28-1b7e-48bc-9ba6-09af1a533545 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:05.121526 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3923c28-1b7e-48bc-9ba6-09af1a533545 None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:05.140794 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 261/517] 10.222.0.23 () {56 vars in 1137 bytes} [Thu May 19 11:58:04 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:58:05.142220 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3923c28-1b7e-48bc-9ba6-09af1a533545 None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:05.172511 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:58:05.186574 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1af660-444c-45b0-ad32-f21a455c3b16 None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:05.222612 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c204869-4d2a-4495-8489-260690c6cd7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:05.223424 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c204869-4d2a-4495-8489-260690c6cd7a None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:05.223703 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c204869-4d2a-4495-8489-260690c6cd7a None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:05.224005 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c204869-4d2a-4495-8489-260690c6cd7a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:05.234879 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c204869-4d2a-4495-8489-260690c6cd7a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:05.236386 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c204869-4d2a-4495-8489-260690c6cd7a None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:05.253332 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3923c28-1b7e-48bc-9ba6-09af1a533545 None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:05.256057 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application [None req-4c204869-4d2a-4495-8489-260690c6cd7a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 11:58:05.256057 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 11:58:05.256057 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 11:58:05.256057 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 11:58:05.256057 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 11:58:05.256057 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 11:58:05.256057 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 11:58:05.256057 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 11:58:05.256057 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 11:58:05.256057 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 11:58:05.256057 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 11:58:05.256057 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 11:58:05.256057 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 11:58:05.256057 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 11:58:05.256057 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 11:58:05.256951 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 11:58:05.256951 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 11:58:05.256951 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 11:58:05.256951 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 11:58:05.256951 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 11:58:05.256951 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 11:58:05.256951 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.get_or_create( May 19 11:58:05.256951 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 11:58:05.256951 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application with Lock( May 19 11:58:05.256951 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 11:58:05.256951 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._enter() May 19 11:58:05.256951 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 11:58:05.256951 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 11:58:05.256951 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 11:58:05.256951 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.creator() May 19 11:58:05.256951 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 11:58:05.256951 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application created_value = creator( May 19 11:58:05.257784 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 11:58:05.257784 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 11:58:05.257784 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 11:58:05.257784 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application May 19 11:58:05.257784 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 257/518] 10.222.0.23 () {56 vars in 1108 bytes} [Thu May 19 11:58:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 11:58:05.258046 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c204869-4d2a-4495-8489-260690c6cd7a None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:05.289079 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d325326b-a725-4393-84f5-75ef3d3176b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:05.289797 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d325326b-a725-4393-84f5-75ef3d3176b3 None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:05.290085 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d325326b-a725-4393-84f5-75ef3d3176b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:05.290367 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d325326b-a725-4393-84f5-75ef3d3176b3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:05.297440 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d325326b-a725-4393-84f5-75ef3d3176b3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:05.298929 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d325326b-a725-4393-84f5-75ef3d3176b3 None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:05.310090 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:58:05.313838 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application [None req-d325326b-a725-4393-84f5-75ef3d3176b3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 11:58:05.313838 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 11:58:05.313838 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 11:58:05.313838 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 11:58:05.313838 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 11:58:05.313838 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 11:58:05.313838 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 11:58:05.313838 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 11:58:05.313838 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 11:58:05.313838 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 11:58:05.313838 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 11:58:05.313838 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 11:58:05.313838 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 11:58:05.313838 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 11:58:05.313838 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 11:58:05.314880 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 11:58:05.314880 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 11:58:05.314880 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 11:58:05.314880 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 11:58:05.314880 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 11:58:05.314880 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 11:58:05.314880 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.get_or_create( May 19 11:58:05.314880 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 11:58:05.314880 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application with Lock( May 19 11:58:05.314880 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 11:58:05.314880 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._enter() May 19 11:58:05.314880 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 11:58:05.314880 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 11:58:05.314880 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 11:58:05.314880 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.creator() May 19 11:58:05.314880 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 11:58:05.314880 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application created_value = creator( May 19 11:58:05.315747 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 11:58:05.315747 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 11:58:05.315747 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 11:58:05.315747 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 11:58:05.315747 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 11:58:05.315747 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 11:58:05.315747 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 11:58:05.315747 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application May 19 11:58:05.315747 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 262/519] 10.222.0.23 () {56 vars in 1111 bytes} [Thu May 19 11:58:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 11:58:05.316392 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d325326b-a725-4393-84f5-75ef3d3176b3 None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:05.366602 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c204869-4d2a-4495-8489-260690c6cd7a None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:05.402275 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac788cf2-0a16-42e3-9745-03ae53928806 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:05.402937 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac788cf2-0a16-42e3-9745-03ae53928806 None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:05.403218 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac788cf2-0a16-42e3-9745-03ae53928806 None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:05.403500 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac788cf2-0a16-42e3-9745-03ae53928806 None admin] PATH_INFO: `/v3/domains` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:05.404264 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac788cf2-0a16-42e3-9745-03ae53928806 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 11:58:05.405186 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac788cf2-0a16-42e3-9745-03ae53928806 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:05.406584 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac788cf2-0a16-42e3-9745-03ae53928806 None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:05.407777 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:58:05.418035 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 258/520] 10.222.0.23 () {56 vars in 1109 bytes} [Thu May 19 11:58:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:58:05.418851 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac788cf2-0a16-42e3-9745-03ae53928806 None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:05.420241 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d325326b-a725-4393-84f5-75ef3d3176b3 None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:05.458480 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55767eea-cc0a-435c-a9bf-45d406f4ca29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:05.459155 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55767eea-cc0a-435c-a9bf-45d406f4ca29 None admin] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:05.459443 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55767eea-cc0a-435c-a9bf-45d406f4ca29 None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:05.459723 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55767eea-cc0a-435c-a9bf-45d406f4ca29 None admin] PATH_INFO: `/v3/users` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:05.460497 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55767eea-cc0a-435c-a9bf-45d406f4ca29 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:05.461968 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55767eea-cc0a-435c-a9bf-45d406f4ca29 None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:05.520658 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac788cf2-0a16-42e3-9745-03ae53928806 None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:05.527703 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:58:05.548272 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 263/521] 10.222.0.23 () {60 vars in 1132 bytes} [Thu May 19 11:58:05 2022] POST /identity/v3/users => generated 247 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 11:58:05.548272 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55767eea-cc0a-435c-a9bf-45d406f4ca29 None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:05.556186 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbf61d6-ac90-4f2f-9fd2-bb1db064ed59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:05.556864 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbf61d6-ac90-4f2f-9fd2-bb1db064ed59 None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:05.557146 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbf61d6-ac90-4f2f-9fd2-bb1db064ed59 None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:05.557421 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbf61d6-ac90-4f2f-9fd2-bb1db064ed59 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:05.643163 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bbf61d6-ac90-4f2f-9fd2-bb1db064ed59 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:05.645278 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bbf61d6-ac90-4f2f-9fd2-bb1db064ed59 None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:05.673186 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55767eea-cc0a-435c-a9bf-45d406f4ca29 None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:05.713287 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0f9264-e5fc-4c31-840c-7bec240aef11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:05.713981 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0f9264-e5fc-4c31-840c-7bec240aef11 None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:05.714269 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0f9264-e5fc-4c31-840c-7bec240aef11 None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:05.714656 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0f9264-e5fc-4c31-840c-7bec240aef11 None admin] PATH_INFO: `/v3/domains` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:05.715410 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0f9264-e5fc-4c31-840c-7bec240aef11 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 11:58:05.715725 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0f9264-e5fc-4c31-840c-7bec240aef11 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:05.718085 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0f9264-e5fc-4c31-840c-7bec240aef11 None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:05.731358 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 264/522] 10.222.0.23 () {56 vars in 1109 bytes} [Thu May 19 11:58:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:58:05.732153 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0f9264-e5fc-4c31-840c-7bec240aef11 None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:05.736533 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:58:05.832151 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 259/523] 10.222.0.23 () {58 vars in 1282 bytes} [Thu May 19 11:58:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 414 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 11:58:05.833436 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbf61d6-ac90-4f2f-9fd2-bb1db064ed59 None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:05.841629 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:58:05.910154 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0f9264-e5fc-4c31-840c-7bec240aef11 None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:05.943457 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbf61d6-ac90-4f2f-9fd2-bb1db064ed59 None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:05.954309 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334ec590-a690-4434-987d-b53cfcaa33d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:05.954999 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334ec590-a690-4434-987d-b53cfcaa33d2 None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:05.955363 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334ec590-a690-4434-987d-b53cfcaa33d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:05.955646 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334ec590-a690-4434-987d-b53cfcaa33d2 None admin] PATH_INFO: `/v3/projects` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:05.957000 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334ec590-a690-4434-987d-b53cfcaa33d2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 11:58:05.957000 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334ec590-a690-4434-987d-b53cfcaa33d2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:05.958319 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334ec590-a690-4434-987d-b53cfcaa33d2 None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:05.978227 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 265/524] 10.222.0.23 () {56 vars in 1148 bytes} [Thu May 19 11:58:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:58:05.979086 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334ec590-a690-4434-987d-b53cfcaa33d2 None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:05.992912 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc1f613-70b9-4b0e-b642-869a9321bcb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:05.993713 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc1f613-70b9-4b0e-b642-869a9321bcb6 None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:05.994191 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc1f613-70b9-4b0e-b642-869a9321bcb6 None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:05.994307 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc1f613-70b9-4b0e-b642-869a9321bcb6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:06.001578 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc1f613-70b9-4b0e-b642-869a9321bcb6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:06.003030 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc1f613-70b9-4b0e-b642-869a9321bcb6 None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:06.018754 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application [None req-5dc1f613-70b9-4b0e-b642-869a9321bcb6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 11:58:06.018754 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 11:58:06.018754 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 11:58:06.018754 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 11:58:06.018754 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 11:58:06.018754 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 11:58:06.018754 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 11:58:06.018754 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 11:58:06.018754 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 11:58:06.018754 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 11:58:06.018754 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 11:58:06.018754 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 11:58:06.018754 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 11:58:06.018754 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 11:58:06.018754 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 11:58:06.019669 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 11:58:06.019669 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 11:58:06.019669 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 11:58:06.019669 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 11:58:06.019669 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 11:58:06.019669 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 11:58:06.019669 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.get_or_create( May 19 11:58:06.019669 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 11:58:06.019669 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application with Lock( May 19 11:58:06.019669 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 11:58:06.019669 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._enter() May 19 11:58:06.019669 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 11:58:06.019669 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 11:58:06.019669 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 11:58:06.019669 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.creator() May 19 11:58:06.019669 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 11:58:06.019669 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application created_value = creator( May 19 11:58:06.020710 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 11:58:06.020710 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 11:58:06.020710 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 11:58:06.020710 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 11:58:06.020710 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 11:58:06.020710 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 11:58:06.020710 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 11:58:06.020710 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application May 19 11:58:06.024724 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 260/525] 10.222.0.23 () {56 vars in 1111 bytes} [Thu May 19 11:58:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 11:58:06.025663 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc1f613-70b9-4b0e-b642-869a9321bcb6 None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:06.054633 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:58:06.097471 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334ec590-a690-4434-987d-b53cfcaa33d2 None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:06.100538 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:58:06.138527 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc1f613-70b9-4b0e-b642-869a9321bcb6 None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:06.141896 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44766ab6-6983-489d-ad46-e5b9c7262297 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:06.142904 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44766ab6-6983-489d-ad46-e5b9c7262297 None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:06.143339 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44766ab6-6983-489d-ad46-e5b9c7262297 None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:06.143759 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44766ab6-6983-489d-ad46-e5b9c7262297 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:06.152385 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44766ab6-6983-489d-ad46-e5b9c7262297 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:06.154064 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44766ab6-6983-489d-ad46-e5b9c7262297 None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:06.172011 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application [None req-44766ab6-6983-489d-ad46-e5b9c7262297 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. May 19 11:58:06.172011 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 11:58:06.172011 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 11:58:06.172011 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 11:58:06.172011 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 11:58:06.172011 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 11:58:06.172011 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 11:58:06.172011 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 11:58:06.172011 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 11:58:06.172011 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 11:58:06.172011 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 11:58:06.172011 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 11:58:06.172011 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 11:58:06.172011 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 11:58:06.172011 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 11:58:06.172011 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 11:58:06.173050 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 11:58:06.173050 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 11:58:06.173050 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 11:58:06.173050 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 11:58:06.173050 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 11:58:06.173050 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 11:58:06.173050 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 11:58:06.173050 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 11:58:06.173050 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 11:58:06.173050 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.get_or_create( May 19 11:58:06.173050 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 11:58:06.173050 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application with Lock( May 19 11:58:06.173050 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 11:58:06.173050 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._enter() May 19 11:58:06.173050 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 11:58:06.173050 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 11:58:06.173050 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 11:58:06.173050 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.creator() May 19 11:58:06.174043 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 11:58:06.174043 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application created_value = creator( May 19 11:58:06.174043 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 11:58:06.174043 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 11:58:06.174043 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 11:58:06.174043 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 11:58:06.174043 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 11:58:06.174043 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 11:58:06.174043 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. May 19 11:58:06.174043 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application May 19 11:58:06.175948 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 266/526] 10.222.0.23 () {56 vars in 1102 bytes} [Thu May 19 11:58:05 2022] GET /identity/v3/users/neutron => generated 85 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 11:58:06.176909 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44766ab6-6983-489d-ad46-e5b9c7262297 None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:06.182061 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f969cb-cfab-408e-a63c-8d1168435719 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:06.182872 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f969cb-cfab-408e-a63c-8d1168435719 None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:06.183154 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f969cb-cfab-408e-a63c-8d1168435719 None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:06.183430 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f969cb-cfab-408e-a63c-8d1168435719 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:06.196506 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f969cb-cfab-408e-a63c-8d1168435719 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:06.197967 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f969cb-cfab-408e-a63c-8d1168435719 None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:06.216453 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application [None req-01f969cb-cfab-408e-a63c-8d1168435719 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 11:58:06.216453 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 11:58:06.216453 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 11:58:06.216453 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 11:58:06.216453 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 11:58:06.216453 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 11:58:06.216453 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 11:58:06.216453 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 11:58:06.216453 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 11:58:06.216453 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 11:58:06.216453 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 11:58:06.216453 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 11:58:06.216453 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 11:58:06.216453 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 11:58:06.216453 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 11:58:06.218137 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 11:58:06.218137 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 11:58:06.218137 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 11:58:06.218137 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 11:58:06.218137 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 11:58:06.218137 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 11:58:06.218137 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.get_or_create( May 19 11:58:06.218137 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 11:58:06.218137 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application with Lock( May 19 11:58:06.218137 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 11:58:06.218137 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._enter() May 19 11:58:06.218137 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 11:58:06.218137 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 11:58:06.218137 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 11:58:06.218137 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.creator() May 19 11:58:06.218137 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 11:58:06.218137 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application created_value = creator( May 19 11:58:06.219023 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 11:58:06.219023 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 11:58:06.219023 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 11:58:06.219023 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application May 19 11:58:06.219023 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 261/527] 10.222.0.23 () {56 vars in 1108 bytes} [Thu May 19 11:58:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 11:58:06.219654 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f969cb-cfab-408e-a63c-8d1168435719 None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:06.305567 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44766ab6-6983-489d-ad46-e5b9c7262297 None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:06.333245 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:58:06.341516 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f969cb-cfab-408e-a63c-8d1168435719 None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:06.350787 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60eba569-06b8-4786-b04b-0ff05ed046f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:06.351816 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60eba569-06b8-4786-b04b-0ff05ed046f1 None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:06.352345 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60eba569-06b8-4786-b04b-0ff05ed046f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:06.352834 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60eba569-06b8-4786-b04b-0ff05ed046f1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:06.354410 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60eba569-06b8-4786-b04b-0ff05ed046f1 None admin] RBAC: Adding query filter params (role.id=7c59061ba7064a0db6032433fb0eec6b, scope.project.id=d70d25bf69314820b8aaecb65576978a, user.id=34a98d75d84b4c67981377e0f26eca62) {{(pid=80540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 11:58:06.354976 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60eba569-06b8-4786-b04b-0ff05ed046f1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:06.357819 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60eba569-06b8-4786-b04b-0ff05ed046f1 None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:06.362810 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:58:06.380204 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 267/528] 10.222.0.23 () {56 vars in 1374 bytes} [Thu May 19 11:58:06 2022] GET /identity/v3/role_assignments?user.id=34a98d75d84b4c67981377e0f26eca62&scope.project.id=d70d25bf69314820b8aaecb65576978a&role.id=7c59061ba7064a0db6032433fb0eec6b => generated 259 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:58:06.381109 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6d2d51-a207-4a7e-8b21-664c82e93a86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:06.381683 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60eba569-06b8-4786-b04b-0ff05ed046f1 None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:06.382344 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6d2d51-a207-4a7e-8b21-664c82e93a86 None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:06.382938 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6d2d51-a207-4a7e-8b21-664c82e93a86 None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:06.383638 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6d2d51-a207-4a7e-8b21-664c82e93a86 None admin] PATH_INFO: `/v3/projects` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:06.384573 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6d2d51-a207-4a7e-8b21-664c82e93a86 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 11:58:06.384983 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6d2d51-a207-4a7e-8b21-664c82e93a86 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:06.386689 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6d2d51-a207-4a7e-8b21-664c82e93a86 None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:06.413351 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 262/529] 10.222.0.23 () {56 vars in 1148 bytes} [Thu May 19 11:58:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:58:06.414507 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6d2d51-a207-4a7e-8b21-664c82e93a86 None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:06.540451 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60eba569-06b8-4786-b04b-0ff05ed046f1 None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:06.542862 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6d2d51-a207-4a7e-8b21-664c82e93a86 None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:06.586410 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1ba790-775b-49ee-941b-bf6e475af9ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:06.587713 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1ba790-775b-49ee-941b-bf6e475af9ea None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:06.588261 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1ba790-775b-49ee-941b-bf6e475af9ea None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:06.588691 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1ba790-775b-49ee-941b-bf6e475af9ea None admin] PATH_INFO: `/v3/users` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:06.589822 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1ba790-775b-49ee-941b-bf6e475af9ea None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 11:58:06.590390 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1ba790-775b-49ee-941b-bf6e475af9ea None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:06.591394 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80eb6fff-5faa-44f7-b472-782a1b5b0348 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:06.592307 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80eb6fff-5faa-44f7-b472-782a1b5b0348 None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:06.592692 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80eb6fff-5faa-44f7-b472-782a1b5b0348 None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:06.592898 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1ba790-775b-49ee-941b-bf6e475af9ea None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:06.593145 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80eb6fff-5faa-44f7-b472-782a1b5b0348 None admin] PATH_INFO: `/v3/domains` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:06.593984 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80eb6fff-5faa-44f7-b472-782a1b5b0348 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 11:58:06.594387 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80eb6fff-5faa-44f7-b472-782a1b5b0348 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:06.595899 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80eb6fff-5faa-44f7-b472-782a1b5b0348 None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:06.612763 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 263/530] 10.222.0.23 () {56 vars in 1109 bytes} [Thu May 19 11:58:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:58:06.613552 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80eb6fff-5faa-44f7-b472-782a1b5b0348 None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:06.620586 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 268/531] 10.222.0.23 () {56 vars in 1139 bytes} [Thu May 19 11:58:06 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:58:06.621783 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1ba790-775b-49ee-941b-bf6e475af9ea None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:06.667419 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:58:06.678201 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:58:06.747331 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80eb6fff-5faa-44f7-b472-782a1b5b0348 None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:06.753557 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1ba790-775b-49ee-941b-bf6e475af9ea None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:06.790277 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d691713-66e3-4fa1-8cb3-54f786392648 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:06.791058 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d691713-66e3-4fa1-8cb3-54f786392648 None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:06.791321 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d691713-66e3-4fa1-8cb3-54f786392648 None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:06.791607 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d691713-66e3-4fa1-8cb3-54f786392648 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:06.792595 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d691713-66e3-4fa1-8cb3-54f786392648 None admin] RBAC: Adding query filter params (role.id=7c59061ba7064a0db6032433fb0eec6b, scope.project.id=d70d25bf69314820b8aaecb65576978a, user.id=8800254c0e784e42a791fb9abc274d79) {{(pid=80539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 11:58:06.792918 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d691713-66e3-4fa1-8cb3-54f786392648 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:06.794376 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d691713-66e3-4fa1-8cb3-54f786392648 None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:06.801071 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 264/532] 10.222.0.23 () {56 vars in 1374 bytes} [Thu May 19 11:58:06 2022] GET /identity/v3/role_assignments?user.id=8800254c0e784e42a791fb9abc274d79&scope.project.id=d70d25bf69314820b8aaecb65576978a&role.id=7c59061ba7064a0db6032433fb0eec6b => generated 259 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:58:06.802100 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d691713-66e3-4fa1-8cb3-54f786392648 None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:06.802379 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6b7fca-054a-4658-834e-e16761cc57c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:06.803358 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6b7fca-054a-4658-834e-e16761cc57c1 None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:06.803831 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6b7fca-054a-4658-834e-e16761cc57c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:06.804250 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6b7fca-054a-4658-834e-e16761cc57c1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:06.814972 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6b7fca-054a-4658-834e-e16761cc57c1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:06.817718 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6b7fca-054a-4658-834e-e16761cc57c1 None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:06.834332 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application [None req-8c6b7fca-054a-4658-834e-e16761cc57c1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 11:58:06.834332 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 11:58:06.834332 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 11:58:06.834332 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 11:58:06.834332 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 11:58:06.834332 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 11:58:06.834332 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 11:58:06.834332 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 11:58:06.834332 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 11:58:06.834332 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 11:58:06.834332 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 11:58:06.834332 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 11:58:06.834332 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 11:58:06.834332 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 11:58:06.834332 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 11:58:06.835341 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 11:58:06.835341 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 11:58:06.835341 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 11:58:06.835341 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 11:58:06.835341 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 11:58:06.835341 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 11:58:06.835341 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.get_or_create( May 19 11:58:06.835341 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 11:58:06.835341 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application with Lock( May 19 11:58:06.835341 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 11:58:06.835341 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._enter() May 19 11:58:06.835341 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 11:58:06.835341 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 11:58:06.835341 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 11:58:06.835341 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.creator() May 19 11:58:06.835341 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 11:58:06.835341 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application created_value = creator( May 19 11:58:06.836238 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 11:58:06.836238 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 11:58:06.836238 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 11:58:06.836238 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 11:58:06.836238 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 11:58:06.836238 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 11:58:06.836238 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 11:58:06.836238 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application May 19 11:58:06.841470 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 269/533] 10.222.0.23 () {56 vars in 1111 bytes} [Thu May 19 11:58:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 11:58:06.846547 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6b7fca-054a-4658-834e-e16761cc57c1 None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:07.042511 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6b7fca-054a-4658-834e-e16761cc57c1 None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:07.052104 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:58:07.082205 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d691713-66e3-4fa1-8cb3-54f786392648 None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:07.092738 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:58:07.093173 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39cd95e-01b0-4d86-828c-e762d6fdfe04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:07.093943 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39cd95e-01b0-4d86-828c-e762d6fdfe04 None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:07.094233 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39cd95e-01b0-4d86-828c-e762d6fdfe04 None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:07.094516 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39cd95e-01b0-4d86-828c-e762d6fdfe04 None admin] PATH_INFO: `/v3/projects` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:07.095322 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39cd95e-01b0-4d86-828c-e762d6fdfe04 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 11:58:07.095638 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39cd95e-01b0-4d86-828c-e762d6fdfe04 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:07.097150 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39cd95e-01b0-4d86-828c-e762d6fdfe04 None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:07.121778 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 270/534] 10.222.0.23 () {56 vars in 1148 bytes} [Thu May 19 11:58:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:58:07.128222 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39cd95e-01b0-4d86-828c-e762d6fdfe04 None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:07.158779 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8090662-5fd4-49e9-9565-e7f5c2bc75ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:07.159503 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8090662-5fd4-49e9-9565-e7f5c2bc75ef None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:07.159815 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8090662-5fd4-49e9-9565-e7f5c2bc75ef None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:07.160097 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8090662-5fd4-49e9-9565-e7f5c2bc75ef None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:07.319257 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39cd95e-01b0-4d86-828c-e762d6fdfe04 None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:07.328656 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8090662-5fd4-49e9-9565-e7f5c2bc75ef None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:07.330169 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8090662-5fd4-49e9-9565-e7f5c2bc75ef None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:07.359525 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc27b1d-8655-406e-a88a-93013fc40dcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:07.360331 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc27b1d-8655-406e-a88a-93013fc40dcf None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:07.360798 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc27b1d-8655-406e-a88a-93013fc40dcf None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:07.361090 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc27b1d-8655-406e-a88a-93013fc40dcf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:07.361938 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc27b1d-8655-406e-a88a-93013fc40dcf None admin] RBAC: Adding query filter params (role.id=7c59061ba7064a0db6032433fb0eec6b, scope.project.id=d70d25bf69314820b8aaecb65576978a, user.id=21697802f2be42f9baa8e07490d5ebdd) {{(pid=80540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 11:58:07.362260 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc27b1d-8655-406e-a88a-93013fc40dcf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:07.363754 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc27b1d-8655-406e-a88a-93013fc40dcf None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:07.370861 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 271/535] 10.222.0.23 () {56 vars in 1374 bytes} [Thu May 19 11:58:07 2022] GET /identity/v3/role_assignments?user.id=21697802f2be42f9baa8e07490d5ebdd&scope.project.id=d70d25bf69314820b8aaecb65576978a&role.id=7c59061ba7064a0db6032433fb0eec6b => generated 259 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 11:58:07.532497 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 265/536] 10.222.0.23 () {58 vars in 1282 bytes} [Thu May 19 11:58:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 731 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 11:58:07.538163 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc27b1d-8655-406e-a88a-93013fc40dcf None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:07.596546 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:58:07.735837 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc27b1d-8655-406e-a88a-93013fc40dcf None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:07.826290 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452d0c3c-33e6-4eb0-a92a-f6f0286491cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:07.828171 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452d0c3c-33e6-4eb0-a92a-f6f0286491cf None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:07.828703 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452d0c3c-33e6-4eb0-a92a-f6f0286491cf None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:07.829180 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452d0c3c-33e6-4eb0-a92a-f6f0286491cf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:07.847000 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452d0c3c-33e6-4eb0-a92a-f6f0286491cf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:07.848947 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452d0c3c-33e6-4eb0-a92a-f6f0286491cf None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:07.879680 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application [None req-452d0c3c-33e6-4eb0-a92a-f6f0286491cf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 11:58:07.879680 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 11:58:07.879680 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 11:58:07.879680 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 11:58:07.879680 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 11:58:07.879680 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 11:58:07.879680 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 11:58:07.879680 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 11:58:07.879680 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 11:58:07.879680 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 11:58:07.879680 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 11:58:07.879680 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 11:58:07.879680 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 11:58:07.879680 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 11:58:07.879680 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 11:58:07.880646 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 11:58:07.880646 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 11:58:07.880646 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 11:58:07.880646 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 11:58:07.880646 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 11:58:07.880646 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 11:58:07.880646 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.get_or_create( May 19 11:58:07.880646 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 11:58:07.880646 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application with Lock( May 19 11:58:07.880646 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 11:58:07.880646 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._enter() May 19 11:58:07.880646 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 11:58:07.880646 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 11:58:07.880646 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 11:58:07.880646 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.creator() May 19 11:58:07.880646 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 11:58:07.880646 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application created_value = creator( May 19 11:58:07.881462 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 11:58:07.881462 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 11:58:07.881462 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 11:58:07.881462 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application May 19 11:58:07.883434 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 272/537] 10.222.0.23 () {56 vars in 1108 bytes} [Thu May 19 11:58:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 346 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 11:58:07.888881 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8090662-5fd4-49e9-9565-e7f5c2bc75ef None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:07.891872 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:58:08.057233 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8090662-5fd4-49e9-9565-e7f5c2bc75ef None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:08.060689 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24be55da-3156-4427-85cf-3710ee624713 None None] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:08.061805 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24be55da-3156-4427-85cf-3710ee624713 None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:08.061805 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24be55da-3156-4427-85cf-3710ee624713 None None] PATH_INFO: `/` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:08.064739 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 273/538] 10.222.0.23 () {54 vars in 926 bytes} [Thu May 19 11:58:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 11:58:08.068985 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2551bc-751f-4c37-a2e6-4d43377d6b65 None None] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:08.069280 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2551bc-751f-4c37-a2e6-4d43377d6b65 None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:08.069560 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2551bc-751f-4c37-a2e6-4d43377d6b65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:08.087976 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.auth.core [None req-9a2551bc-751f-4c37-a2e6-4d43377d6b65 None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 11:58:08.089442 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-9a2551bc-751f-4c37-a2e6-4d43377d6b65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 11:58:08.137371 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9256541b-6ff4-48cd-8e16-7e15f21e57ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:08.138717 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9256541b-6ff4-48cd-8e16-7e15f21e57ee None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:08.139754 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9256541b-6ff4-48cd-8e16-7e15f21e57ee None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:08.140335 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9256541b-6ff4-48cd-8e16-7e15f21e57ee None admin] PATH_INFO: `/v3/domains` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:08.141323 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9256541b-6ff4-48cd-8e16-7e15f21e57ee None admin] RBAC: Adding query filter params (name=Default) {{(pid=80539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 11:58:08.141865 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9256541b-6ff4-48cd-8e16-7e15f21e57ee None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:08.143510 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9256541b-6ff4-48cd-8e16-7e15f21e57ee None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:08.159457 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 266/539] 10.222.0.23 () {56 vars in 1109 bytes} [Thu May 19 11:58:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:58:08.164806 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9256541b-6ff4-48cd-8e16-7e15f21e57ee None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:08.193108 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 274/540] 10.222.0.23 () {58 vars in 1023 bytes} [Thu May 19 11:58:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 11:58:08.236664 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2551bc-751f-4c37-a2e6-4d43377d6b65 None None] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:08.255924 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:58:08.256507 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:58:08.307820 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9256541b-6ff4-48cd-8e16-7e15f21e57ee None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:08.373972 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbf528e-fde4-4d8d-aba0-1cf885af9b53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:08.374778 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbf528e-fde4-4d8d-aba0-1cf885af9b53 None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:08.375060 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbf528e-fde4-4d8d-aba0-1cf885af9b53 None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:08.375337 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbf528e-fde4-4d8d-aba0-1cf885af9b53 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:08.392743 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fbf528e-fde4-4d8d-aba0-1cf885af9b53 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:08.394136 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fbf528e-fde4-4d8d-aba0-1cf885af9b53 None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:08.416554 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application [None req-2fbf528e-fde4-4d8d-aba0-1cf885af9b53 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 11:58:08.416554 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 11:58:08.416554 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 11:58:08.416554 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 11:58:08.416554 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 11:58:08.416554 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 11:58:08.416554 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 11:58:08.416554 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 11:58:08.416554 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 11:58:08.416554 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 11:58:08.416554 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 11:58:08.416554 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 11:58:08.416554 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 11:58:08.416554 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 11:58:08.416554 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 11:58:08.417460 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 11:58:08.417460 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 11:58:08.417460 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 11:58:08.417460 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 11:58:08.417460 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 11:58:08.417460 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 11:58:08.417460 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.get_or_create( May 19 11:58:08.417460 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 11:58:08.417460 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application with Lock( May 19 11:58:08.417460 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 11:58:08.417460 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._enter() May 19 11:58:08.417460 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 11:58:08.417460 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 11:58:08.417460 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 11:58:08.417460 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.creator() May 19 11:58:08.417460 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 11:58:08.417460 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application created_value = creator( May 19 11:58:08.418287 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 11:58:08.418287 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 11:58:08.418287 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 11:58:08.418287 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 11:58:08.418287 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 11:58:08.418287 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 11:58:08.418287 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 11:58:08.418287 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application May 19 11:58:08.418287 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 267/541] 10.222.0.23 () {56 vars in 1111 bytes} [Thu May 19 11:58:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 11:58:08.430906 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbf528e-fde4-4d8d-aba0-1cf885af9b53 None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:08.449593 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2551bc-751f-4c37-a2e6-4d43377d6b65 None None] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:08.523219 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36ed309-42c8-4f99-9251-2fd0e2bf6bfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:08.524062 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36ed309-42c8-4f99-9251-2fd0e2bf6bfb None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:08.524353 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36ed309-42c8-4f99-9251-2fd0e2bf6bfb None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:08.524634 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36ed309-42c8-4f99-9251-2fd0e2bf6bfb None admin] PATH_INFO: `/v3/roles/service` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:08.550171 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36ed309-42c8-4f99-9251-2fd0e2bf6bfb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 11:58:08.558065 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application [None req-f36ed309-42c8-4f99-9251-2fd0e2bf6bfb None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 11:58:08.558065 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 11:58:08.558065 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 11:58:08.558065 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 11:58:08.558065 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 11:58:08.558065 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 11:58:08.558065 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 11:58:08.558065 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 11:58:08.558065 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 11:58:08.558065 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 11:58:08.558065 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 11:58:08.558065 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 11:58:08.558065 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 11:58:08.558065 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 11:58:08.558065 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 11:58:08.558065 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 11:58:08.558985 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 11:58:08.558985 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 11:58:08.558985 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 11:58:08.558985 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 11:58:08.558985 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 11:58:08.558985 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 11:58:08.558985 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 11:58:08.558985 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 11:58:08.558985 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 11:58:08.558985 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.get_or_create( May 19 11:58:08.558985 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 11:58:08.558985 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application with Lock( May 19 11:58:08.558985 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 11:58:08.558985 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._enter() May 19 11:58:08.558985 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 11:58:08.558985 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 11:58:08.559771 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 11:58:08.559771 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.creator() May 19 11:58:08.559771 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 11:58:08.559771 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application created_value = creator( May 19 11:58:08.559771 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 11:58:08.559771 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 11:58:08.559771 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 11:58:08.559771 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 11:58:08.559771 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 11:58:08.559771 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 11:58:08.559771 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 11:58:08.559771 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application May 19 11:58:08.559771 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 275/542] 10.222.0.23 () {56 vars in 1102 bytes} [Thu May 19 11:58:08 2022] GET /identity/v3/roles/service => generated 85 bytes in 323 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 11:58:08.571298 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36ed309-42c8-4f99-9251-2fd0e2bf6bfb None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:08.638455 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbf528e-fde4-4d8d-aba0-1cf885af9b53 None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:08.690515 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:58:08.694721 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:58:08.709695 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab6a797-2355-4eaf-86dc-50638ec9ec28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:08.712040 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab6a797-2355-4eaf-86dc-50638ec9ec28 None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:08.712040 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab6a797-2355-4eaf-86dc-50638ec9ec28 None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:08.712040 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab6a797-2355-4eaf-86dc-50638ec9ec28 None admin] PATH_INFO: `/v3/projects` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:08.712504 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab6a797-2355-4eaf-86dc-50638ec9ec28 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 11:58:08.713007 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab6a797-2355-4eaf-86dc-50638ec9ec28 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:08.714790 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab6a797-2355-4eaf-86dc-50638ec9ec28 None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:08.727649 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 268/543] 10.222.0.23 () {56 vars in 1148 bytes} [Thu May 19 11:58:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:58:08.733067 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab6a797-2355-4eaf-86dc-50638ec9ec28 None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:08.753337 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36ed309-42c8-4f99-9251-2fd0e2bf6bfb None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:08.828613 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c407a3db-a0ea-4b4b-967f-97418f90cfe9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:08.829312 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c407a3db-a0ea-4b4b-967f-97418f90cfe9 None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:08.829610 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c407a3db-a0ea-4b4b-967f-97418f90cfe9 None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:08.829893 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c407a3db-a0ea-4b4b-967f-97418f90cfe9 None admin] PATH_INFO: `/v3/roles` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:08.830677 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c407a3db-a0ea-4b4b-967f-97418f90cfe9 None admin] RBAC: Adding query filter params (name=service) {{(pid=80540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 11:58:08.830993 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c407a3db-a0ea-4b4b-967f-97418f90cfe9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:08.832460 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c407a3db-a0ea-4b4b-967f-97418f90cfe9 None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:08.847760 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 276/544] 10.222.0.23 () {56 vars in 1103 bytes} [Thu May 19 11:58:08 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:58:08.853241 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c407a3db-a0ea-4b4b-967f-97418f90cfe9 None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:08.899885 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab6a797-2355-4eaf-86dc-50638ec9ec28 None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:08.978037 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ad55d8-9070-47df-9cb5-635b4d62c0ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:08.978724 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ad55d8-9070-47df-9cb5-635b4d62c0ba None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:08.979012 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ad55d8-9070-47df-9cb5-635b4d62c0ba None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:08.979291 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ad55d8-9070-47df-9cb5-635b4d62c0ba None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:08.980161 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ad55d8-9070-47df-9cb5-635b4d62c0ba None admin] RBAC: Adding query filter params (role.id=7c59061ba7064a0db6032433fb0eec6b, scope.project.id=d70d25bf69314820b8aaecb65576978a, user.id=1a00b3fa7497456895e5e0bf105c4fc6) {{(pid=80539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 11:58:08.980532 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ad55d8-9070-47df-9cb5-635b4d62c0ba None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:08.982000 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ad55d8-9070-47df-9cb5-635b4d62c0ba None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:08.989012 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 269/545] 10.222.0.23 () {56 vars in 1374 bytes} [Thu May 19 11:58:08 2022] GET /identity/v3/role_assignments?user.id=1a00b3fa7497456895e5e0bf105c4fc6&scope.project.id=d70d25bf69314820b8aaecb65576978a&role.id=7c59061ba7064a0db6032433fb0eec6b => generated 259 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 11:58:08.990485 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac427bd-d93b-4912-b054-ac8d27868bfc None None] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:08.990777 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac427bd-d93b-4912-b054-ac8d27868bfc None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:08.991056 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac427bd-d93b-4912-b054-ac8d27868bfc None None] PATH_INFO: `/` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:08.992740 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 270/546] 10.222.0.23 () {54 vars in 926 bytes} [Thu May 19 11:58:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 11:58:09.000559 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72daa1a-f83b-42be-8590-46436877f385 None None] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:09.001079 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72daa1a-f83b-42be-8590-46436877f385 None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:09.001483 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72daa1a-f83b-42be-8590-46436877f385 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:09.015154 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c407a3db-a0ea-4b4b-967f-97418f90cfe9 None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:09.025445 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.auth.core [None req-f72daa1a-f83b-42be-8590-46436877f385 None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 11:58:09.031384 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-f72daa1a-f83b-42be-8590-46436877f385 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 11:58:09.100487 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a628923-9259-4958-80a7-353b23468e7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:09.101313 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a628923-9259-4958-80a7-353b23468e7d None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:09.101597 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a628923-9259-4958-80a7-353b23468e7d None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:09.101874 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a628923-9259-4958-80a7-353b23468e7d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:09.113292 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:58:09.119403 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:58:09.134454 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a628923-9259-4958-80a7-353b23468e7d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:09.135999 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a628923-9259-4958-80a7-353b23468e7d None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:09.165899 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application [None req-1a628923-9259-4958-80a7-353b23468e7d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 11:58:09.165899 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 11:58:09.165899 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 11:58:09.165899 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 11:58:09.165899 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 11:58:09.165899 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 11:58:09.165899 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 11:58:09.165899 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 11:58:09.165899 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 11:58:09.165899 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 11:58:09.165899 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 11:58:09.165899 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 11:58:09.165899 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 11:58:09.165899 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 11:58:09.165899 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 11:58:09.166812 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 11:58:09.166812 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 11:58:09.166812 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 11:58:09.166812 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 11:58:09.166812 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 11:58:09.166812 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 11:58:09.166812 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.get_or_create( May 19 11:58:09.166812 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 11:58:09.166812 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application with Lock( May 19 11:58:09.166812 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 11:58:09.166812 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._enter() May 19 11:58:09.166812 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 11:58:09.166812 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 11:58:09.166812 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 11:58:09.166812 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.creator() May 19 11:58:09.166812 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 11:58:09.166812 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application created_value = creator( May 19 11:58:09.168612 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 11:58:09.168612 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 11:58:09.168612 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 11:58:09.168612 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application May 19 11:58:09.168612 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 277/547] 10.222.0.23 () {56 vars in 1108 bytes} [Thu May 19 11:58:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 315 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 11:58:09.173294 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a628923-9259-4958-80a7-353b23468e7d None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:09.177520 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 271/548] 10.222.0.23 () {58 vars in 1023 bytes} [Thu May 19 11:58:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 11:58:09.234690 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72daa1a-f83b-42be-8590-46436877f385 None None] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:09.310636 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a628923-9259-4958-80a7-353b23468e7d None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:09.356290 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec74b29-4d2d-4360-9707-ff1d038b76ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:09.356970 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec74b29-4d2d-4360-9707-ff1d038b76ec None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:09.357264 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec74b29-4d2d-4360-9707-ff1d038b76ec None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:09.357572 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec74b29-4d2d-4360-9707-ff1d038b76ec None admin] PATH_INFO: `/v3/domains` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:09.359159 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec74b29-4d2d-4360-9707-ff1d038b76ec None admin] RBAC: Adding query filter params (name=Default) {{(pid=80540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 11:58:09.359476 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec74b29-4d2d-4360-9707-ff1d038b76ec None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:09.361023 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec74b29-4d2d-4360-9707-ff1d038b76ec None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:09.370756 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72daa1a-f83b-42be-8590-46436877f385 None None] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:09.381156 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 278/549] 10.222.0.23 () {56 vars in 1109 bytes} [Thu May 19 11:58:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:58:09.393004 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec74b29-4d2d-4360-9707-ff1d038b76ec None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:09.418297 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1f38a1-33e2-42ca-87fd-7e1f1c672b45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:09.419580 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1f38a1-33e2-42ca-87fd-7e1f1c672b45 None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:09.420163 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1f38a1-33e2-42ca-87fd-7e1f1c672b45 None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:09.420713 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1f38a1-33e2-42ca-87fd-7e1f1c672b45 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:09.431749 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1f38a1-33e2-42ca-87fd-7e1f1c672b45 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 11:58:09.441055 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application [None req-5b1f38a1-33e2-42ca-87fd-7e1f1c672b45 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 11:58:09.441055 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 11:58:09.441055 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 11:58:09.441055 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 11:58:09.441055 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 11:58:09.441055 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 11:58:09.441055 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 11:58:09.441055 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 11:58:09.441055 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 11:58:09.441055 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 11:58:09.441055 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 11:58:09.441055 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 11:58:09.441055 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 11:58:09.441055 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 11:58:09.441055 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 11:58:09.441055 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 11:58:09.441953 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 11:58:09.441953 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 11:58:09.441953 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 11:58:09.441953 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 11:58:09.441953 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 11:58:09.441953 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 11:58:09.441953 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 11:58:09.441953 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 11:58:09.441953 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 11:58:09.441953 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.get_or_create( May 19 11:58:09.441953 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 11:58:09.441953 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application with Lock( May 19 11:58:09.441953 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 11:58:09.441953 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._enter() May 19 11:58:09.441953 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 11:58:09.441953 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 11:58:09.442740 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 11:58:09.442740 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.creator() May 19 11:58:09.442740 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 11:58:09.442740 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application created_value = creator( May 19 11:58:09.442740 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 11:58:09.442740 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 11:58:09.442740 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 11:58:09.442740 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 11:58:09.442740 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 11:58:09.442740 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 11:58:09.442740 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 11:58:09.442740 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application May 19 11:58:09.445797 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 272/550] 10.222.0.23 () {56 vars in 1102 bytes} [Thu May 19 11:58:09 2022] GET /identity/v3/roles/service => generated 85 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 11:58:09.452591 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1f38a1-33e2-42ca-87fd-7e1f1c672b45 None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:09.471703 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:58:09.503535 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:58:09.568868 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec74b29-4d2d-4360-9707-ff1d038b76ec None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:09.580807 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1f38a1-33e2-42ca-87fd-7e1f1c672b45 None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:09.611066 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b061fd-c7c4-40e8-a018-b2b6b06efdc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:09.612385 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b061fd-c7c4-40e8-a018-b2b6b06efdc5 None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:09.612935 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b061fd-c7c4-40e8-a018-b2b6b06efdc5 None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:09.613421 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b061fd-c7c4-40e8-a018-b2b6b06efdc5 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:09.621053 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85016ef9-f81c-41b5-b9f9-0eef159f40a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:09.621757 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85016ef9-f81c-41b5-b9f9-0eef159f40a4 None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:09.622015 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85016ef9-f81c-41b5-b9f9-0eef159f40a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:09.622287 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85016ef9-f81c-41b5-b9f9-0eef159f40a4 None admin] PATH_INFO: `/v3/roles` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:09.623047 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85016ef9-f81c-41b5-b9f9-0eef159f40a4 None admin] RBAC: Adding query filter params (name=service) {{(pid=80539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 11:58:09.624015 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85016ef9-f81c-41b5-b9f9-0eef159f40a4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:09.624519 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b061fd-c7c4-40e8-a018-b2b6b06efdc5 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:09.625684 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85016ef9-f81c-41b5-b9f9-0eef159f40a4 None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:09.626473 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b061fd-c7c4-40e8-a018-b2b6b06efdc5 None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:09.635359 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 273/551] 10.222.0.23 () {56 vars in 1103 bytes} [Thu May 19 11:58:09 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 11:58:09.641022 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d332a3ef-e576-4d45-8e72-a77a5b0a414d None None] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:09.641468 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d332a3ef-e576-4d45-8e72-a77a5b0a414d None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:09.641833 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d332a3ef-e576-4d45-8e72-a77a5b0a414d None None] PATH_INFO: `/` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:09.643556 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application [None req-e0b061fd-c7c4-40e8-a018-b2b6b06efdc5 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 19 11:58:09.643556 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 11:58:09.643556 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 11:58:09.643556 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 11:58:09.643556 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 11:58:09.643556 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 11:58:09.643556 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 11:58:09.643556 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 11:58:09.643556 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 11:58:09.643556 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 11:58:09.643556 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 11:58:09.643556 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 11:58:09.643556 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 11:58:09.643556 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 11:58:09.643556 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 11:58:09.643556 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 11:58:09.648709 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 11:58:09.648709 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 11:58:09.648709 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 11:58:09.648709 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 11:58:09.648709 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 11:58:09.648709 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 11:58:09.648709 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 11:58:09.648709 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 11:58:09.648709 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 11:58:09.648709 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.get_or_create( May 19 11:58:09.648709 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 11:58:09.648709 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application with Lock( May 19 11:58:09.648709 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 11:58:09.648709 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._enter() May 19 11:58:09.648709 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 11:58:09.648709 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 11:58:09.648709 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 11:58:09.648709 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.creator() May 19 11:58:09.649693 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 11:58:09.649693 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application created_value = creator( May 19 11:58:09.649693 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 11:58:09.649693 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 11:58:09.649693 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 11:58:09.649693 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 11:58:09.649693 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 11:58:09.649693 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 11:58:09.649693 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 19 11:58:09.649693 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application May 19 11:58:09.650225 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 274/552] 10.222.0.23 () {54 vars in 926 bytes} [Thu May 19 11:58:09 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 11:58:09.650225 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d332a3ef-e576-4d45-8e72-a77a5b0a414d None None] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:09.652634 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 279/553] 10.222.0.23 () {56 vars in 1096 bytes} [Thu May 19 11:58:09 2022] GET /identity/v3/users/swift => generated 83 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 11:58:09.654221 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb9b367-7883-4a69-b433-148aab3485ca None None] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:09.654822 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb9b367-7883-4a69-b433-148aab3485ca None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:09.655340 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb9b367-7883-4a69-b433-148aab3485ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:09.679427 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.auth.core [None req-ceb9b367-7883-4a69-b433-148aab3485ca None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 11:58:09.681261 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-ceb9b367-7883-4a69-b433-148aab3485ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 11:58:09.767228 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 280/554] 10.222.0.23 () {58 vars in 1023 bytes} [Thu May 19 11:58:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 11:58:09.767870 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:58:09.769623 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb9b367-7883-4a69-b433-148aab3485ca None None] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:09.775169 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d332a3ef-e576-4d45-8e72-a77a5b0a414d None None] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:09.827671 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:58:09.840982 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a96551-01c4-46df-bb53-a23e33d534e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:09.843282 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a96551-01c4-46df-bb53-a23e33d534e3 None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:09.843581 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a96551-01c4-46df-bb53-a23e33d534e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:09.843885 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a96551-01c4-46df-bb53-a23e33d534e3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:09.859163 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a96551-01c4-46df-bb53-a23e33d534e3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:09.860763 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a96551-01c4-46df-bb53-a23e33d534e3 None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:09.881196 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application [None req-e1a96551-01c4-46df-bb53-a23e33d534e3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 11:58:09.881196 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 11:58:09.881196 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 11:58:09.881196 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 11:58:09.881196 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 11:58:09.881196 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 11:58:09.881196 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 11:58:09.881196 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 11:58:09.881196 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 11:58:09.881196 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 11:58:09.881196 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 11:58:09.881196 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 11:58:09.881196 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 11:58:09.881196 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 11:58:09.881196 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 11:58:09.882126 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 11:58:09.882126 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 11:58:09.882126 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 11:58:09.882126 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 11:58:09.882126 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 11:58:09.882126 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 11:58:09.882126 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.get_or_create( May 19 11:58:09.882126 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 11:58:09.882126 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application with Lock( May 19 11:58:09.882126 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 11:58:09.882126 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._enter() May 19 11:58:09.882126 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 11:58:09.882126 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 11:58:09.882126 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 11:58:09.882126 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.creator() May 19 11:58:09.882126 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 11:58:09.882126 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application created_value = creator( May 19 11:58:09.882960 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 11:58:09.882960 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 11:58:09.882960 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 11:58:09.882960 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application May 19 11:58:09.882960 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 275/555] 10.222.0.23 () {56 vars in 1108 bytes} [Thu May 19 11:58:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 11:58:09.884002 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a96551-01c4-46df-bb53-a23e33d534e3 None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:09.929960 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb9b367-7883-4a69-b433-148aab3485ca None None] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:09.974962 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb11a3a-03b5-4135-bc34-afb8255bbd68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:09.975854 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb11a3a-03b5-4135-bc34-afb8255bbd68 None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:09.976140 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb11a3a-03b5-4135-bc34-afb8255bbd68 None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:09.976492 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb11a3a-03b5-4135-bc34-afb8255bbd68 None admin] PATH_INFO: `/v3/users` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:09.977436 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb11a3a-03b5-4135-bc34-afb8255bbd68 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 11:58:09.977824 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb11a3a-03b5-4135-bc34-afb8255bbd68 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:09.979364 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb11a3a-03b5-4135-bc34-afb8255bbd68 None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:10.003983 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 281/556] 10.222.0.23 () {56 vars in 1135 bytes} [Thu May 19 11:58:09 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:58:10.005044 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb11a3a-03b5-4135-bc34-afb8255bbd68 None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:10.025690 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a96551-01c4-46df-bb53-a23e33d534e3 None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:10.064149 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283801b4-f812-4ca2-9b2f-e3f47aaf997f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:10.065260 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283801b4-f812-4ca2-9b2f-e3f47aaf997f None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:10.065442 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283801b4-f812-4ca2-9b2f-e3f47aaf997f None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:10.065841 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283801b4-f812-4ca2-9b2f-e3f47aaf997f None admin] PATH_INFO: `/v3/roles/service` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:10.078371 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283801b4-f812-4ca2-9b2f-e3f47aaf997f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 11:58:10.086746 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application [None req-283801b4-f812-4ca2-9b2f-e3f47aaf997f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 11:58:10.086746 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 11:58:10.086746 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 11:58:10.086746 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 11:58:10.086746 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 11:58:10.086746 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 11:58:10.086746 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 11:58:10.086746 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 11:58:10.086746 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 11:58:10.086746 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 11:58:10.086746 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 11:58:10.086746 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 11:58:10.086746 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 11:58:10.086746 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 11:58:10.086746 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 11:58:10.086746 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 11:58:10.087723 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 11:58:10.087723 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 11:58:10.087723 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 11:58:10.087723 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 11:58:10.087723 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 11:58:10.087723 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 11:58:10.087723 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 11:58:10.087723 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 11:58:10.087723 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 11:58:10.087723 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.get_or_create( May 19 11:58:10.087723 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 11:58:10.087723 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application with Lock( May 19 11:58:10.087723 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 11:58:10.087723 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._enter() May 19 11:58:10.087723 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 11:58:10.087723 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 11:58:10.089386 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 11:58:10.089386 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.creator() May 19 11:58:10.089386 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 11:58:10.089386 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application created_value = creator( May 19 11:58:10.089386 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 11:58:10.089386 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 11:58:10.089386 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 11:58:10.089386 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 11:58:10.089386 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 11:58:10.089386 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 11:58:10.089386 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 11:58:10.089386 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application May 19 11:58:10.093963 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 276/557] 10.222.0.23 () {56 vars in 1102 bytes} [Thu May 19 11:58:09 2022] GET /identity/v3/roles/service => generated 85 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 11:58:10.095342 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283801b4-f812-4ca2-9b2f-e3f47aaf997f None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:10.111544 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:58:10.141916 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:58:10.150287 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb11a3a-03b5-4135-bc34-afb8255bbd68 None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:10.189171 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc500b3-b137-4d49-ba44-41d8e107253a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:10.190015 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc500b3-b137-4d49-ba44-41d8e107253a None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:10.190489 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc500b3-b137-4d49-ba44-41d8e107253a None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:10.190776 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc500b3-b137-4d49-ba44-41d8e107253a None admin] PATH_INFO: `/v3/domains` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:10.191669 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc500b3-b137-4d49-ba44-41d8e107253a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 11:58:10.192425 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc500b3-b137-4d49-ba44-41d8e107253a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:10.193714 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc500b3-b137-4d49-ba44-41d8e107253a None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:10.211072 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 282/558] 10.222.0.23 () {56 vars in 1109 bytes} [Thu May 19 11:58:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:58:10.211072 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc500b3-b137-4d49-ba44-41d8e107253a None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:10.224604 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283801b4-f812-4ca2-9b2f-e3f47aaf997f None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:10.266407 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a35696f-27ab-4e6e-a0a1-e00e1ed261cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:10.267084 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a35696f-27ab-4e6e-a0a1-e00e1ed261cf None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:10.267477 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a35696f-27ab-4e6e-a0a1-e00e1ed261cf None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:10.267928 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a35696f-27ab-4e6e-a0a1-e00e1ed261cf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:10.335947 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc500b3-b137-4d49-ba44-41d8e107253a None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:10.341748 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a35696f-27ab-4e6e-a0a1-e00e1ed261cf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:10.344021 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a35696f-27ab-4e6e-a0a1-e00e1ed261cf None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:10.355392 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:58:10.373667 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8c662b-6448-40ca-8285-74646424db56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:10.374482 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8c662b-6448-40ca-8285-74646424db56 None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:10.374829 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8c662b-6448-40ca-8285-74646424db56 None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:10.375115 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8c662b-6448-40ca-8285-74646424db56 None admin] PATH_INFO: `/v3/roles` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:10.375913 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8c662b-6448-40ca-8285-74646424db56 None admin] RBAC: Adding query filter params (name=service) {{(pid=80540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 11:58:10.376229 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8c662b-6448-40ca-8285-74646424db56 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:10.377668 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8c662b-6448-40ca-8285-74646424db56 None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:10.392133 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 283/559] 10.222.0.23 () {56 vars in 1103 bytes} [Thu May 19 11:58:10 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:58:10.393730 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8c662b-6448-40ca-8285-74646424db56 None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:10.416934 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:58:10.464086 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 277/560] 10.222.0.23 () {58 vars in 1282 bytes} [Thu May 19 11:58:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 11:58:10.464086 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3756af-220f-409b-8a08-2fbfd6a2f0a4 None None] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:10.464086 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3756af-220f-409b-8a08-2fbfd6a2f0a4 None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:10.464086 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3756af-220f-409b-8a08-2fbfd6a2f0a4 None None] PATH_INFO: `/` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:10.464086 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 278/561] 10.222.0.23 () {54 vars in 926 bytes} [Thu May 19 11:58:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 11:58:10.464086 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3756af-220f-409b-8a08-2fbfd6a2f0a4 None None] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:10.515914 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8c662b-6448-40ca-8285-74646424db56 None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:10.554715 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b482f60e-81b3-4837-a6c0-0ea7090f8d05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:10.555513 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b482f60e-81b3-4837-a6c0-0ea7090f8d05 None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:10.555813 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b482f60e-81b3-4837-a6c0-0ea7090f8d05 None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:10.556095 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b482f60e-81b3-4837-a6c0-0ea7090f8d05 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:10.564379 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b482f60e-81b3-4837-a6c0-0ea7090f8d05 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:10.565834 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b482f60e-81b3-4837-a6c0-0ea7090f8d05 None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:10.580757 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application [None req-b482f60e-81b3-4837-a6c0-0ea7090f8d05 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 19 11:58:10.580757 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 11:58:10.580757 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 11:58:10.580757 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 11:58:10.580757 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 11:58:10.580757 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 11:58:10.580757 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 11:58:10.580757 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 11:58:10.580757 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 11:58:10.580757 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 11:58:10.580757 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 11:58:10.580757 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 11:58:10.580757 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 11:58:10.580757 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 11:58:10.580757 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 11:58:10.580757 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 11:58:10.581749 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 11:58:10.581749 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 11:58:10.581749 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 11:58:10.581749 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 11:58:10.581749 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 11:58:10.581749 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 11:58:10.581749 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 11:58:10.581749 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 11:58:10.581749 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 11:58:10.581749 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.get_or_create( May 19 11:58:10.581749 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 11:58:10.581749 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application with Lock( May 19 11:58:10.581749 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 11:58:10.581749 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._enter() May 19 11:58:10.581749 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 11:58:10.581749 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 11:58:10.581749 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 11:58:10.581749 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.creator() May 19 11:58:10.582685 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 11:58:10.582685 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application created_value = creator( May 19 11:58:10.582685 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 11:58:10.582685 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 11:58:10.582685 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 11:58:10.582685 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 11:58:10.582685 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 11:58:10.582685 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 11:58:10.582685 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 19 11:58:10.582685 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application May 19 11:58:10.582685 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 284/562] 10.222.0.23 () {56 vars in 1093 bytes} [Thu May 19 11:58:10 2022] GET /identity/v3/users/nova => generated 82 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 11:58:10.582685 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b482f60e-81b3-4837-a6c0-0ea7090f8d05 None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:10.583429 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3756af-220f-409b-8a08-2fbfd6a2f0a4 None None] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:10.596682 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:58:10.620552 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74669800-66f8-4a79-b32b-ebf4df354c03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:10.621343 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74669800-66f8-4a79-b32b-ebf4df354c03 None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:10.621624 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74669800-66f8-4a79-b32b-ebf4df354c03 None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:10.621901 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74669800-66f8-4a79-b32b-ebf4df354c03 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:10.635095 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74669800-66f8-4a79-b32b-ebf4df354c03 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:10.637044 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74669800-66f8-4a79-b32b-ebf4df354c03 None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:10.656627 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application [None req-74669800-66f8-4a79-b32b-ebf4df354c03 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 11:58:10.656627 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 11:58:10.656627 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 11:58:10.656627 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 11:58:10.656627 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 11:58:10.656627 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 11:58:10.656627 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 11:58:10.656627 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 11:58:10.656627 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 11:58:10.656627 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 11:58:10.656627 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 11:58:10.656627 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 11:58:10.656627 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 11:58:10.656627 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 11:58:10.656627 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 11:58:10.657594 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 11:58:10.657594 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 11:58:10.657594 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 11:58:10.657594 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 11:58:10.657594 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 11:58:10.657594 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 11:58:10.657594 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.get_or_create( May 19 11:58:10.657594 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 11:58:10.657594 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application with Lock( May 19 11:58:10.657594 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 11:58:10.657594 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._enter() May 19 11:58:10.657594 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 11:58:10.657594 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 11:58:10.657594 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 11:58:10.657594 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.creator() May 19 11:58:10.657594 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 11:58:10.657594 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application created_value = creator( May 19 11:58:10.658451 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 11:58:10.658451 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 11:58:10.658451 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 11:58:10.658451 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application May 19 11:58:10.658451 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 279/563] 10.222.0.23 () {56 vars in 1108 bytes} [Thu May 19 11:58:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 11:58:10.659295 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55173a13-78ae-4fa5-ae90-a283e0adfbd6 None None] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:10.659395 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55173a13-78ae-4fa5-ae90-a283e0adfbd6 None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:10.659604 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55173a13-78ae-4fa5-ae90-a283e0adfbd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:10.671268 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:58:10.677531 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.auth.core [None req-55173a13-78ae-4fa5-ae90-a283e0adfbd6 None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 11:58:10.678740 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-55173a13-78ae-4fa5-ae90-a283e0adfbd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 11:58:10.695429 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b482f60e-81b3-4837-a6c0-0ea7090f8d05 None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:10.743624 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c010f4e8-2619-4121-ad67-dffcdbde542f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:10.744497 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c010f4e8-2619-4121-ad67-dffcdbde542f None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:10.744780 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c010f4e8-2619-4121-ad67-dffcdbde542f None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:10.745065 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c010f4e8-2619-4121-ad67-dffcdbde542f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:10.752997 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 280/564] 10.222.0.23 () {58 vars in 1023 bytes} [Thu May 19 11:58:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 11:58:10.753740 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55173a13-78ae-4fa5-ae90-a283e0adfbd6 None None] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:10.764492 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c010f4e8-2619-4121-ad67-dffcdbde542f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:10.767412 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c010f4e8-2619-4121-ad67-dffcdbde542f None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:10.790018 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application [None req-c010f4e8-2619-4121-ad67-dffcdbde542f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 11:58:10.790018 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 11:58:10.790018 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 11:58:10.790018 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 11:58:10.790018 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 11:58:10.790018 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 11:58:10.790018 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 11:58:10.790018 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 11:58:10.790018 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 11:58:10.790018 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 11:58:10.790018 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 11:58:10.790018 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 11:58:10.790018 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 11:58:10.790018 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 11:58:10.790018 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 11:58:10.790979 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 11:58:10.790979 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 11:58:10.790979 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 11:58:10.790979 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 11:58:10.790979 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 11:58:10.790979 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 11:58:10.790979 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.get_or_create( May 19 11:58:10.790979 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 11:58:10.790979 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application with Lock( May 19 11:58:10.790979 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 11:58:10.790979 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._enter() May 19 11:58:10.790979 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 11:58:10.790979 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 11:58:10.790979 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 11:58:10.790979 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.creator() May 19 11:58:10.790979 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 11:58:10.790979 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application created_value = creator( May 19 11:58:10.792576 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 11:58:10.792576 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 11:58:10.792576 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 11:58:10.792576 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application May 19 11:58:10.792576 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 285/565] 10.222.0.23 () {56 vars in 1108 bytes} [Thu May 19 11:58:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 11:58:10.792576 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c010f4e8-2619-4121-ad67-dffcdbde542f None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:10.861927 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:58:10.871693 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55173a13-78ae-4fa5-ae90-a283e0adfbd6 None None] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:10.899729 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c010f4e8-2619-4121-ad67-dffcdbde542f None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:10.912328 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91a3d58-8962-4e99-8e7c-eccca1582eec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:10.913090 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91a3d58-8962-4e99-8e7c-eccca1582eec None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:10.913515 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91a3d58-8962-4e99-8e7c-eccca1582eec None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:10.913857 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91a3d58-8962-4e99-8e7c-eccca1582eec None admin] PATH_INFO: `/v3/users` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:10.914973 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b91a3d58-8962-4e99-8e7c-eccca1582eec None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 11:58:10.915361 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b91a3d58-8962-4e99-8e7c-eccca1582eec None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:10.916943 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b91a3d58-8962-4e99-8e7c-eccca1582eec None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:10.929533 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:58:10.940365 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd9708a-ac78-42c6-b48a-090137fde11f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:10.941284 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd9708a-ac78-42c6-b48a-090137fde11f None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:10.941667 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd9708a-ac78-42c6-b48a-090137fde11f None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:10.942029 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd9708a-ac78-42c6-b48a-090137fde11f None admin] PATH_INFO: `/v3/domains` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:10.942870 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd9708a-ac78-42c6-b48a-090137fde11f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 11:58:10.943251 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd9708a-ac78-42c6-b48a-090137fde11f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:10.947295 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd9708a-ac78-42c6-b48a-090137fde11f None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:10.962669 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 286/566] 10.222.0.23 () {56 vars in 1109 bytes} [Thu May 19 11:58:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:58:10.962669 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd9708a-ac78-42c6-b48a-090137fde11f None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:10.970312 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 281/567] 10.222.0.23 () {56 vars in 1133 bytes} [Thu May 19 11:58:10 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:58:10.971297 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91a3d58-8962-4e99-8e7c-eccca1582eec None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:11.069490 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd9708a-ac78-42c6-b48a-090137fde11f None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:11.077872 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91a3d58-8962-4e99-8e7c-eccca1582eec None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:11.110806 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568ef5b8-8224-436e-9423-4efa60cffedb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:11.111607 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568ef5b8-8224-436e-9423-4efa60cffedb None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:11.112032 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568ef5b8-8224-436e-9423-4efa60cffedb None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:11.112224 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568ef5b8-8224-436e-9423-4efa60cffedb None admin] PATH_INFO: `/v3/roles/service` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:11.115297 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4730dfb-1750-4c9f-a8c1-4a47cecd4d73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:11.116000 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4730dfb-1750-4c9f-a8c1-4a47cecd4d73 None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:11.116293 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4730dfb-1750-4c9f-a8c1-4a47cecd4d73 None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:11.116575 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4730dfb-1750-4c9f-a8c1-4a47cecd4d73 None admin] PATH_INFO: `/v3/domains` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:11.117314 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4730dfb-1750-4c9f-a8c1-4a47cecd4d73 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 11:58:11.117628 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4730dfb-1750-4c9f-a8c1-4a47cecd4d73 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:11.119019 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4730dfb-1750-4c9f-a8c1-4a47cecd4d73 None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:11.123044 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568ef5b8-8224-436e-9423-4efa60cffedb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 11:58:11.131572 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 282/568] 10.222.0.23 () {56 vars in 1109 bytes} [Thu May 19 11:58:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:58:11.131742 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application [None req-568ef5b8-8224-436e-9423-4efa60cffedb None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 11:58:11.131742 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 11:58:11.131742 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 11:58:11.131742 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 11:58:11.131742 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 11:58:11.131742 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 11:58:11.131742 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 11:58:11.131742 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 11:58:11.131742 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 11:58:11.131742 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 11:58:11.131742 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 11:58:11.131742 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 11:58:11.131742 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 11:58:11.131742 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 11:58:11.131742 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 11:58:11.131742 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 11:58:11.132630 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 11:58:11.132630 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 11:58:11.132630 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 11:58:11.132630 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 11:58:11.132630 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 11:58:11.132630 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 11:58:11.132630 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 11:58:11.132630 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 11:58:11.132630 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 11:58:11.132630 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.get_or_create( May 19 11:58:11.132630 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 11:58:11.132630 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application with Lock( May 19 11:58:11.132630 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 11:58:11.132630 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._enter() May 19 11:58:11.132630 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 11:58:11.132630 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 11:58:11.133466 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 11:58:11.133466 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.creator() May 19 11:58:11.133466 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 11:58:11.133466 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application created_value = creator( May 19 11:58:11.133466 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 11:58:11.133466 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 11:58:11.133466 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 11:58:11.133466 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 11:58:11.133466 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 11:58:11.133466 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 11:58:11.133466 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 11:58:11.133466 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application May 19 11:58:11.134718 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 287/569] 10.222.0.23 () {56 vars in 1102 bytes} [Thu May 19 11:58:10 2022] GET /identity/v3/roles/service => generated 85 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 11:58:11.136617 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4730dfb-1750-4c9f-a8c1-4a47cecd4d73 None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:11.136916 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568ef5b8-8224-436e-9423-4efa60cffedb None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:11.151513 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:58:11.194847 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:58:11.250726 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4730dfb-1750-4c9f-a8c1-4a47cecd4d73 None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:11.256671 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568ef5b8-8224-436e-9423-4efa60cffedb None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:11.290898 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16897595-1810-48f5-bea3-92b339316bac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:11.291696 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16897595-1810-48f5-bea3-92b339316bac None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:11.292276 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16897595-1810-48f5-bea3-92b339316bac None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:11.292276 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16897595-1810-48f5-bea3-92b339316bac None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:11.297316 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f760f8-f1a3-40ed-bea1-91350a75cc45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:11.298121 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f760f8-f1a3-40ed-bea1-91350a75cc45 None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:11.298412 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f760f8-f1a3-40ed-bea1-91350a75cc45 None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:11.298693 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f760f8-f1a3-40ed-bea1-91350a75cc45 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:11.300850 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16897595-1810-48f5-bea3-92b339316bac None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:11.302312 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16897595-1810-48f5-bea3-92b339316bac None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:11.310951 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f760f8-f1a3-40ed-bea1-91350a75cc45 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:11.312891 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f760f8-f1a3-40ed-bea1-91350a75cc45 None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:11.317738 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application [None req-16897595-1810-48f5-bea3-92b339316bac None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 19 11:58:11.317738 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 11:58:11.317738 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 11:58:11.317738 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 11:58:11.317738 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 11:58:11.317738 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 11:58:11.317738 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 11:58:11.317738 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 11:58:11.317738 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 11:58:11.317738 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 11:58:11.317738 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 11:58:11.317738 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 11:58:11.317738 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 11:58:11.317738 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 11:58:11.317738 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 11:58:11.317738 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 11:58:11.318659 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 11:58:11.318659 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 11:58:11.318659 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 11:58:11.318659 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 11:58:11.318659 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 11:58:11.318659 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 11:58:11.318659 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 11:58:11.318659 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 11:58:11.318659 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 11:58:11.318659 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.get_or_create( May 19 11:58:11.318659 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 11:58:11.318659 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application with Lock( May 19 11:58:11.318659 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 11:58:11.318659 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._enter() May 19 11:58:11.318659 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 11:58:11.318659 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 11:58:11.318659 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 11:58:11.318659 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.creator() May 19 11:58:11.319615 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 11:58:11.319615 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application created_value = creator( May 19 11:58:11.319615 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 11:58:11.319615 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 11:58:11.319615 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 11:58:11.319615 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 11:58:11.319615 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 11:58:11.319615 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 11:58:11.319615 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 19 11:58:11.319615 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application May 19 11:58:11.319615 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 283/570] 10.222.0.23 () {56 vars in 1099 bytes} [Thu May 19 11:58:11 2022] GET /identity/v3/users/cinder => generated 84 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 11:58:11.320570 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16897595-1810-48f5-bea3-92b339316bac None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:11.336740 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application [None req-05f760f8-f1a3-40ed-bea1-91350a75cc45 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 11:58:11.336740 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 11:58:11.336740 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 11:58:11.336740 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 11:58:11.336740 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 11:58:11.336740 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 11:58:11.336740 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 11:58:11.336740 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 11:58:11.336740 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 11:58:11.336740 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 11:58:11.336740 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 11:58:11.336740 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 11:58:11.336740 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 11:58:11.336740 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 11:58:11.336740 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 11:58:11.337763 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 11:58:11.337763 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 11:58:11.337763 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 11:58:11.337763 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 11:58:11.337763 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 11:58:11.337763 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 11:58:11.337763 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.get_or_create( May 19 11:58:11.337763 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 11:58:11.337763 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application with Lock( May 19 11:58:11.337763 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 11:58:11.337763 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._enter() May 19 11:58:11.337763 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 11:58:11.337763 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 11:58:11.337763 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 11:58:11.337763 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.creator() May 19 11:58:11.337763 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 11:58:11.337763 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application created_value = creator( May 19 11:58:11.338700 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 11:58:11.338700 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 11:58:11.338700 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 11:58:11.338700 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application May 19 11:58:11.338700 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 288/571] 10.222.0.23 () {56 vars in 1108 bytes} [Thu May 19 11:58:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 11:58:11.339053 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f760f8-f1a3-40ed-bea1-91350a75cc45 None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:11.408595 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:58:11.442137 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16897595-1810-48f5-bea3-92b339316bac None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:11.449093 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:58:11.452919 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f760f8-f1a3-40ed-bea1-91350a75cc45 None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:11.478029 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59618b73-d966-45ce-b2c1-587165e83867 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:11.478876 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59618b73-d966-45ce-b2c1-587165e83867 None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:11.479112 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59618b73-d966-45ce-b2c1-587165e83867 None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:11.479391 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59618b73-d966-45ce-b2c1-587165e83867 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:11.486346 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59618b73-d966-45ce-b2c1-587165e83867 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:11.487379 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce97f82-a341-451f-a22b-8988b4be1606 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:11.487808 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59618b73-d966-45ce-b2c1-587165e83867 None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:11.488097 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce97f82-a341-451f-a22b-8988b4be1606 None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:11.488389 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce97f82-a341-451f-a22b-8988b4be1606 None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:11.488671 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce97f82-a341-451f-a22b-8988b4be1606 None admin] PATH_INFO: `/v3/roles` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:11.489417 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce97f82-a341-451f-a22b-8988b4be1606 None admin] RBAC: Adding query filter params (name=service) {{(pid=80540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 11:58:11.489729 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce97f82-a341-451f-a22b-8988b4be1606 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:11.491099 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce97f82-a341-451f-a22b-8988b4be1606 None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:11.500343 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 289/572] 10.222.0.23 () {56 vars in 1103 bytes} [Thu May 19 11:58:11 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:58:11.501153 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce97f82-a341-451f-a22b-8988b4be1606 None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:11.502792 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application [None req-59618b73-d966-45ce-b2c1-587165e83867 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 11:58:11.502792 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 11:58:11.502792 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 11:58:11.502792 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 11:58:11.502792 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 11:58:11.502792 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 11:58:11.502792 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 11:58:11.502792 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 11:58:11.502792 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 11:58:11.502792 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 11:58:11.502792 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 11:58:11.502792 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 11:58:11.502792 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 11:58:11.502792 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 11:58:11.502792 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 11:58:11.503863 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 11:58:11.503863 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 11:58:11.503863 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 11:58:11.503863 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 11:58:11.503863 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 11:58:11.503863 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 11:58:11.503863 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.get_or_create( May 19 11:58:11.503863 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 11:58:11.503863 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application with Lock( May 19 11:58:11.503863 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 11:58:11.503863 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._enter() May 19 11:58:11.503863 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 11:58:11.503863 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 11:58:11.503863 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 11:58:11.503863 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.creator() May 19 11:58:11.503863 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 11:58:11.503863 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application created_value = creator( May 19 11:58:11.504778 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 11:58:11.504778 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 11:58:11.504778 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 11:58:11.504778 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 11:58:11.504778 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 11:58:11.504778 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 11:58:11.504778 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 11:58:11.504778 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application May 19 11:58:11.504778 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 284/573] 10.222.0.23 () {56 vars in 1111 bytes} [Thu May 19 11:58:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 11:58:11.505507 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59618b73-d966-45ce-b2c1-587165e83867 None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:11.605976 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce97f82-a341-451f-a22b-8988b4be1606 None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:11.606215 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59618b73-d966-45ce-b2c1-587165e83867 None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:11.635436 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:58:11.641339 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc197576-fcac-42cb-ab50-dac147618d08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:11.641560 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61bd184f-8cc0-4570-8c54-cc0a720b0d53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:11.642024 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc197576-fcac-42cb-ab50-dac147618d08 None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:11.642218 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bd184f-8cc0-4570-8c54-cc0a720b0d53 None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:11.642311 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc197576-fcac-42cb-ab50-dac147618d08 None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:11.642509 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bd184f-8cc0-4570-8c54-cc0a720b0d53 None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:11.642600 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc197576-fcac-42cb-ab50-dac147618d08 None admin] PATH_INFO: `/v3/users` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:11.642870 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bd184f-8cc0-4570-8c54-cc0a720b0d53 None admin] PATH_INFO: `/v3/domains` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:11.643517 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61bd184f-8cc0-4570-8c54-cc0a720b0d53 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 11:58:11.643633 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc197576-fcac-42cb-ab50-dac147618d08 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 11:58:11.643858 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61bd184f-8cc0-4570-8c54-cc0a720b0d53 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:11.643950 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc197576-fcac-42cb-ab50-dac147618d08 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:11.654283 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61bd184f-8cc0-4570-8c54-cc0a720b0d53 None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:11.654432 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc197576-fcac-42cb-ab50-dac147618d08 None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:11.657380 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 285/574] 10.222.0.23 () {56 vars in 1109 bytes} [Thu May 19 11:58:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:58:11.658735 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e392039-b128-49df-9407-fedec556450c None None] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:11.659071 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e392039-b128-49df-9407-fedec556450c None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:11.659242 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e392039-b128-49df-9407-fedec556450c None None] PATH_INFO: `/` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:11.661050 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 286/575] 10.222.0.23 () {54 vars in 926 bytes} [Thu May 19 11:58:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 11:58:11.661761 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e392039-b128-49df-9407-fedec556450c None None] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:11.667009 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 290/576] 10.222.0.23 () {56 vars in 1137 bytes} [Thu May 19 11:58:11 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:58:11.667009 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc197576-fcac-42cb-ab50-dac147618d08 None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:11.687758 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:58:11.768333 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e392039-b128-49df-9407-fedec556450c None None] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:11.768969 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc197576-fcac-42cb-ab50-dac147618d08 None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:11.802732 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fab21e-404f-419d-9e2f-6a737c940f39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:11.803532 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fab21e-404f-419d-9e2f-6a737c940f39 None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:11.803842 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fab21e-404f-419d-9e2f-6a737c940f39 None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:11.804121 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fab21e-404f-419d-9e2f-6a737c940f39 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:11.804893 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4556190-b456-48c0-b296-3620abb85ac8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:11.805279 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4556190-b456-48c0-b296-3620abb85ac8 None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:11.805569 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4556190-b456-48c0-b296-3620abb85ac8 None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:11.805843 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4556190-b456-48c0-b296-3620abb85ac8 None admin] PATH_INFO: `/v3/projects` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:11.806634 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4556190-b456-48c0-b296-3620abb85ac8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 11:58:11.806946 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4556190-b456-48c0-b296-3620abb85ac8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:11.808443 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4556190-b456-48c0-b296-3620abb85ac8 None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:11.815550 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31fab21e-404f-419d-9e2f-6a737c940f39 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:11.817173 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31fab21e-404f-419d-9e2f-6a737c940f39 None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:11.820977 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 291/577] 10.222.0.23 () {56 vars in 1148 bytes} [Thu May 19 11:58:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:58:11.821791 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4556190-b456-48c0-b296-3620abb85ac8 None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:11.837739 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application [None req-31fab21e-404f-419d-9e2f-6a737c940f39 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 11:58:11.837739 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 11:58:11.837739 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 11:58:11.837739 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 11:58:11.837739 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 11:58:11.837739 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 11:58:11.837739 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 11:58:11.837739 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 11:58:11.837739 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 11:58:11.837739 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 11:58:11.837739 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 11:58:11.837739 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 11:58:11.837739 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 11:58:11.837739 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 11:58:11.837739 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 11:58:11.838809 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 11:58:11.838809 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 11:58:11.838809 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 11:58:11.838809 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 11:58:11.838809 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 11:58:11.838809 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 11:58:11.838809 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.get_or_create( May 19 11:58:11.838809 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 11:58:11.838809 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application with Lock( May 19 11:58:11.838809 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 11:58:11.838809 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._enter() May 19 11:58:11.838809 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 11:58:11.838809 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 11:58:11.838809 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 11:58:11.838809 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.creator() May 19 11:58:11.838809 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 11:58:11.838809 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application created_value = creator( May 19 11:58:11.840024 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 11:58:11.840024 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 11:58:11.840024 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 11:58:11.840024 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application May 19 11:58:11.840024 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 287/578] 10.222.0.23 () {56 vars in 1108 bytes} [Thu May 19 11:58:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 11:58:11.840404 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de196cd9-1638-47a5-bf68-90d82b13c53f None None] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:11.840404 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de196cd9-1638-47a5-bf68-90d82b13c53f None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:11.840680 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de196cd9-1638-47a5-bf68-90d82b13c53f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:11.858080 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.auth.core [None req-de196cd9-1638-47a5-bf68-90d82b13c53f None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 11:58:11.859292 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-de196cd9-1638-47a5-bf68-90d82b13c53f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 11:58:11.894057 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:58:11.909510 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:58:11.926878 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 288/579] 10.222.0.23 () {58 vars in 1023 bytes} [Thu May 19 11:58:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 11:58:11.927753 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de196cd9-1638-47a5-bf68-90d82b13c53f None None] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:11.938277 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4556190-b456-48c0-b296-3620abb85ac8 None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:11.975110 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13acee0-9d8b-40cb-bd51-fe9efe4ad414 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:11.976076 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13acee0-9d8b-40cb-bd51-fe9efe4ad414 None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:11.976465 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13acee0-9d8b-40cb-bd51-fe9efe4ad414 None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:11.976845 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13acee0-9d8b-40cb-bd51-fe9efe4ad414 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:11.984834 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13acee0-9d8b-40cb-bd51-fe9efe4ad414 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:11.986407 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13acee0-9d8b-40cb-bd51-fe9efe4ad414 None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:12.000739 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application [None req-c13acee0-9d8b-40cb-bd51-fe9efe4ad414 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 11:58:12.000739 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 11:58:12.000739 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 11:58:12.000739 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 11:58:12.000739 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 11:58:12.000739 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 11:58:12.000739 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 11:58:12.000739 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 11:58:12.000739 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 11:58:12.000739 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 11:58:12.000739 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 11:58:12.000739 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 11:58:12.000739 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 11:58:12.000739 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 11:58:12.000739 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 11:58:12.001794 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 11:58:12.001794 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 11:58:12.001794 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 11:58:12.001794 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 11:58:12.001794 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 11:58:12.001794 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 11:58:12.001794 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.get_or_create( May 19 11:58:12.001794 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 11:58:12.001794 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application with Lock( May 19 11:58:12.001794 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 11:58:12.001794 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._enter() May 19 11:58:12.001794 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 11:58:12.001794 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 11:58:12.001794 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 11:58:12.001794 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.creator() May 19 11:58:12.001794 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 11:58:12.001794 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application created_value = creator( May 19 11:58:12.002692 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 11:58:12.002692 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 11:58:12.002692 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 11:58:12.002692 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 11:58:12.002692 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 11:58:12.002692 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 11:58:12.002692 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 11:58:12.002692 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application May 19 11:58:12.002692 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 292/580] 10.222.0.23 () {56 vars in 1111 bytes} [Thu May 19 11:58:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 11:58:12.002692 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13acee0-9d8b-40cb-bd51-fe9efe4ad414 None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:12.036025 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de196cd9-1638-47a5-bf68-90d82b13c53f None None] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:12.071112 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13f7899-b400-4c04-b2fd-d02498c62dea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:12.071931 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13f7899-b400-4c04-b2fd-d02498c62dea None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:12.072218 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13f7899-b400-4c04-b2fd-d02498c62dea None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:12.072493 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13f7899-b400-4c04-b2fd-d02498c62dea None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:12.083805 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13f7899-b400-4c04-b2fd-d02498c62dea None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:12.085269 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13f7899-b400-4c04-b2fd-d02498c62dea None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:12.102982 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application [None req-a13f7899-b400-4c04-b2fd-d02498c62dea None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 11:58:12.102982 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 11:58:12.102982 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 11:58:12.102982 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 11:58:12.102982 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 11:58:12.102982 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 11:58:12.102982 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 11:58:12.102982 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 11:58:12.102982 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 11:58:12.102982 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 11:58:12.102982 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 11:58:12.102982 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 11:58:12.102982 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 11:58:12.102982 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 11:58:12.102982 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 11:58:12.104138 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 11:58:12.104138 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 11:58:12.104138 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 11:58:12.104138 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 11:58:12.104138 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 11:58:12.104138 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 11:58:12.104138 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.get_or_create( May 19 11:58:12.104138 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 11:58:12.104138 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application with Lock( May 19 11:58:12.104138 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 11:58:12.104138 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._enter() May 19 11:58:12.104138 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 11:58:12.104138 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 11:58:12.104138 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 11:58:12.104138 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.creator() May 19 11:58:12.104138 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 11:58:12.104138 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application created_value = creator( May 19 11:58:12.105060 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 11:58:12.105060 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 11:58:12.105060 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 11:58:12.105060 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application May 19 11:58:12.105298 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13acee0-9d8b-40cb-bd51-fe9efe4ad414 None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:12.105375 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 289/581] 10.222.0.23 () {56 vars in 1108 bytes} [Thu May 19 11:58:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 11:58:12.105375 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13f7899-b400-4c04-b2fd-d02498c62dea None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:12.132814 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:58:12.141028 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172f79e9-091b-4ad7-875a-5a2e1914ada9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:12.146423 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172f79e9-091b-4ad7-875a-5a2e1914ada9 None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:12.146423 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172f79e9-091b-4ad7-875a-5a2e1914ada9 None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:12.146423 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172f79e9-091b-4ad7-875a-5a2e1914ada9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:12.146423 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172f79e9-091b-4ad7-875a-5a2e1914ada9 None admin] RBAC: Adding query filter params (role.id=7c59061ba7064a0db6032433fb0eec6b, scope.project.id=d70d25bf69314820b8aaecb65576978a, user.id=15cbfc73ae1243d2a8b1e70f9cd99876) {{(pid=80540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 11:58:12.146423 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172f79e9-091b-4ad7-875a-5a2e1914ada9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:12.146423 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172f79e9-091b-4ad7-875a-5a2e1914ada9 None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:12.150912 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 293/582] 10.222.0.23 () {56 vars in 1374 bytes} [Thu May 19 11:58:12 2022] GET /identity/v3/role_assignments?user.id=15cbfc73ae1243d2a8b1e70f9cd99876&scope.project.id=d70d25bf69314820b8aaecb65576978a&role.id=7c59061ba7064a0db6032433fb0eec6b => generated 259 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:58:12.151667 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172f79e9-091b-4ad7-875a-5a2e1914ada9 None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:12.155473 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:58:12.223732 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13f7899-b400-4c04-b2fd-d02498c62dea None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:12.263752 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96798e98-d878-49c3-a140-8657a4c8c6db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:12.265100 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96798e98-d878-49c3-a140-8657a4c8c6db None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:12.265331 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96798e98-d878-49c3-a140-8657a4c8c6db None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:12.265597 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96798e98-d878-49c3-a140-8657a4c8c6db None admin] PATH_INFO: `/v3/domains` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:12.266355 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96798e98-d878-49c3-a140-8657a4c8c6db None admin] RBAC: Adding query filter params (name=Default) {{(pid=80539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 11:58:12.266664 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96798e98-d878-49c3-a140-8657a4c8c6db None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:12.268113 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96798e98-d878-49c3-a140-8657a4c8c6db None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:12.286267 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 290/583] 10.222.0.23 () {56 vars in 1109 bytes} [Thu May 19 11:58:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:58:12.286473 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172f79e9-091b-4ad7-875a-5a2e1914ada9 None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:12.286899 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96798e98-d878-49c3-a140-8657a4c8c6db None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:12.327454 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1c0e8c-2212-4588-bb8c-8def8cb2cab4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:12.328304 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1c0e8c-2212-4588-bb8c-8def8cb2cab4 None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:12.328591 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1c0e8c-2212-4588-bb8c-8def8cb2cab4 None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:12.328870 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1c0e8c-2212-4588-bb8c-8def8cb2cab4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:12.344460 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1c0e8c-2212-4588-bb8c-8def8cb2cab4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 11:58:12.355331 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application [None req-ff1c0e8c-2212-4588-bb8c-8def8cb2cab4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 11:58:12.355331 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 11:58:12.355331 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 11:58:12.355331 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 11:58:12.355331 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 11:58:12.355331 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 11:58:12.355331 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 11:58:12.355331 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 11:58:12.355331 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 11:58:12.355331 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 11:58:12.355331 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 11:58:12.355331 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 11:58:12.355331 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 11:58:12.355331 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 11:58:12.355331 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 11:58:12.355331 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 11:58:12.356412 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 11:58:12.356412 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 11:58:12.356412 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 11:58:12.356412 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 11:58:12.356412 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 11:58:12.356412 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 11:58:12.356412 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 11:58:12.356412 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 11:58:12.356412 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 11:58:12.356412 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.get_or_create( May 19 11:58:12.356412 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 11:58:12.356412 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application with Lock( May 19 11:58:12.356412 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 11:58:12.356412 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._enter() May 19 11:58:12.356412 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 11:58:12.356412 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 11:58:12.357759 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 11:58:12.357759 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.creator() May 19 11:58:12.357759 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 11:58:12.357759 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application created_value = creator( May 19 11:58:12.357759 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 11:58:12.357759 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 11:58:12.357759 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 11:58:12.357759 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 11:58:12.357759 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 11:58:12.357759 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 11:58:12.357759 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 11:58:12.357759 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application May 19 11:58:12.357759 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 294/584] 10.222.0.23 () {56 vars in 1102 bytes} [Thu May 19 11:58:12 2022] GET /identity/v3/roles/service => generated 85 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 11:58:12.358765 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1c0e8c-2212-4588-bb8c-8def8cb2cab4 None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:12.385236 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:58:12.401880 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96798e98-d878-49c3-a140-8657a4c8c6db None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:12.432171 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:58:12.442079 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78987d6-3ca5-4e70-83cf-8487c5acef60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:12.443013 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78987d6-3ca5-4e70-83cf-8487c5acef60 None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:12.443466 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78987d6-3ca5-4e70-83cf-8487c5acef60 None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:12.443976 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78987d6-3ca5-4e70-83cf-8487c5acef60 None admin] PATH_INFO: `/v3/projects` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:12.444959 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78987d6-3ca5-4e70-83cf-8487c5acef60 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 11:58:12.445683 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78987d6-3ca5-4e70-83cf-8487c5acef60 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:12.447292 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78987d6-3ca5-4e70-83cf-8487c5acef60 None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:12.466208 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 291/585] 10.222.0.23 () {56 vars in 1148 bytes} [Thu May 19 11:58:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:58:12.468514 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78987d6-3ca5-4e70-83cf-8487c5acef60 None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:12.474314 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1c0e8c-2212-4588-bb8c-8def8cb2cab4 None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:12.509993 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287b3cb3-919c-4ae6-a319-e40e863a6428 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:12.510664 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287b3cb3-919c-4ae6-a319-e40e863a6428 None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:12.510953 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287b3cb3-919c-4ae6-a319-e40e863a6428 None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:12.511238 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287b3cb3-919c-4ae6-a319-e40e863a6428 None admin] PATH_INFO: `/v3/domains` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:12.511994 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287b3cb3-919c-4ae6-a319-e40e863a6428 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 11:58:12.512310 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287b3cb3-919c-4ae6-a319-e40e863a6428 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:12.513707 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287b3cb3-919c-4ae6-a319-e40e863a6428 None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:12.533329 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 295/586] 10.222.0.23 () {56 vars in 1109 bytes} [Thu May 19 11:58:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:58:12.533329 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287b3cb3-919c-4ae6-a319-e40e863a6428 None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:12.588312 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78987d6-3ca5-4e70-83cf-8487c5acef60 None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:12.624385 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ff3dca-3a1a-4abc-a982-a818497625bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:12.624992 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ff3dca-3a1a-4abc-a982-a818497625bc None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:12.625279 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ff3dca-3a1a-4abc-a982-a818497625bc None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:12.625559 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ff3dca-3a1a-4abc-a982-a818497625bc None admin] PATH_INFO: `/v3/users/glance` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:12.632963 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ff3dca-3a1a-4abc-a982-a818497625bc None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:12.634416 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ff3dca-3a1a-4abc-a982-a818497625bc None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:12.643167 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287b3cb3-919c-4ae6-a319-e40e863a6428 None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:12.650006 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application [None req-01ff3dca-3a1a-4abc-a982-a818497625bc None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. May 19 11:58:12.650006 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 11:58:12.650006 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 11:58:12.650006 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 11:58:12.650006 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 11:58:12.650006 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 11:58:12.650006 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 11:58:12.650006 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 11:58:12.650006 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 11:58:12.650006 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 11:58:12.650006 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 11:58:12.650006 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 11:58:12.650006 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 11:58:12.650006 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 11:58:12.650006 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 11:58:12.650006 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 11:58:12.651055 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 11:58:12.651055 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 11:58:12.651055 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 11:58:12.651055 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 11:58:12.651055 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 11:58:12.651055 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 11:58:12.651055 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 11:58:12.651055 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 11:58:12.651055 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 11:58:12.651055 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.get_or_create( May 19 11:58:12.651055 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 11:58:12.651055 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application with Lock( May 19 11:58:12.651055 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 11:58:12.651055 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._enter() May 19 11:58:12.651055 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 11:58:12.651055 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 11:58:12.651055 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 11:58:12.651055 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.creator() May 19 11:58:12.652171 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 11:58:12.652171 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application created_value = creator( May 19 11:58:12.652171 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 11:58:12.652171 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 11:58:12.652171 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 11:58:12.652171 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 11:58:12.652171 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 11:58:12.652171 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 11:58:12.652171 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. May 19 11:58:12.652171 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application May 19 11:58:12.654081 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 292/587] 10.222.0.23 () {56 vars in 1099 bytes} [Thu May 19 11:58:12 2022] GET /identity/v3/users/glance => generated 84 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 11:58:12.654081 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ff3dca-3a1a-4abc-a982-a818497625bc None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:12.658476 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:58:12.672082 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:58:12.684001 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539fb9cb-7abb-4f03-ad69-d7f2cfd5f734 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:12.684679 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539fb9cb-7abb-4f03-ad69-d7f2cfd5f734 None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:12.684967 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539fb9cb-7abb-4f03-ad69-d7f2cfd5f734 None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:12.685246 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539fb9cb-7abb-4f03-ad69-d7f2cfd5f734 None admin] PATH_INFO: `/v3/roles` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:12.687231 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539fb9cb-7abb-4f03-ad69-d7f2cfd5f734 None admin] RBAC: Adding query filter params (name=service) {{(pid=80540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 11:58:12.687571 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539fb9cb-7abb-4f03-ad69-d7f2cfd5f734 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:12.689520 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539fb9cb-7abb-4f03-ad69-d7f2cfd5f734 None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:12.704441 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 296/588] 10.222.0.23 () {56 vars in 1103 bytes} [Thu May 19 11:58:12 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:58:12.704441 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539fb9cb-7abb-4f03-ad69-d7f2cfd5f734 None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:12.783563 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ff3dca-3a1a-4abc-a982-a818497625bc None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:12.817858 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b938c255-682d-4a27-8d93-508db41dcc81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:12.818755 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b938c255-682d-4a27-8d93-508db41dcc81 None admin] REQUEST_METHOD: `PUT` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:12.819041 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b938c255-682d-4a27-8d93-508db41dcc81 None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:12.819320 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b938c255-682d-4a27-8d93-508db41dcc81 None admin] PATH_INFO: `/v3/projects/d70d25bf69314820b8aaecb65576978a/users/34a98d75d84b4c67981377e0f26eca62/roles/7c59061ba7064a0db6032433fb0eec6b` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:12.821621 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539fb9cb-7abb-4f03-ad69-d7f2cfd5f734 None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:12.835486 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b938c255-682d-4a27-8d93-508db41dcc81 None admin] RBAC: Authorizing `identity:create_grant(project_id=d70d25bf69314820b8aaecb65576978a, user_id=34a98d75d84b4c67981377e0f26eca62, role_id=7c59061ba7064a0db6032433fb0eec6b)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:12.836990 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b938c255-682d-4a27-8d93-508db41dcc81 None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:12.857400 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb27a78-2d39-49bf-b237-9773d2941649 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:12.858743 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb27a78-2d39-49bf-b237-9773d2941649 None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:12.859002 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb27a78-2d39-49bf-b237-9773d2941649 None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:12.859280 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb27a78-2d39-49bf-b237-9773d2941649 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:12.866582 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb27a78-2d39-49bf-b237-9773d2941649 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:12.867974 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb27a78-2d39-49bf-b237-9773d2941649 None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:12.883090 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application [None req-5cb27a78-2d39-49bf-b237-9773d2941649 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 11:58:12.883090 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 11:58:12.883090 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 11:58:12.883090 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 11:58:12.883090 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 11:58:12.883090 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 11:58:12.883090 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 11:58:12.883090 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 11:58:12.883090 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 11:58:12.883090 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 11:58:12.883090 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 11:58:12.883090 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 11:58:12.883090 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 11:58:12.883090 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 11:58:12.883090 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 11:58:12.884009 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 11:58:12.884009 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 11:58:12.884009 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 11:58:12.884009 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 11:58:12.884009 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 11:58:12.884009 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 11:58:12.884009 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.get_or_create( May 19 11:58:12.884009 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 11:58:12.884009 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application with Lock( May 19 11:58:12.884009 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 11:58:12.884009 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._enter() May 19 11:58:12.884009 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 11:58:12.884009 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 11:58:12.884009 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 11:58:12.884009 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.creator() May 19 11:58:12.884009 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 11:58:12.884009 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application created_value = creator( May 19 11:58:12.884885 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 11:58:12.884885 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 11:58:12.884885 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 11:58:12.884885 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 11:58:12.884885 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 11:58:12.884885 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 11:58:12.884885 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 11:58:12.884885 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application May 19 11:58:12.884885 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 297/589] 10.222.0.23 () {56 vars in 1111 bytes} [Thu May 19 11:58:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 11:58:12.885359 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb27a78-2d39-49bf-b237-9773d2941649 None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:12.909334 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 293/590] 10.222.0.23 () {58 vars in 1439 bytes} [Thu May 19 11:58:12 2022] PUT /identity/v3/projects/d70d25bf69314820b8aaecb65576978a/users/34a98d75d84b4c67981377e0f26eca62/roles/7c59061ba7064a0db6032433fb0eec6b => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 11:58:12.909334 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b938c255-682d-4a27-8d93-508db41dcc81 None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:12.954169 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:58:12.987148 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:58:13.025011 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb27a78-2d39-49bf-b237-9773d2941649 None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:13.065426 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b938c255-682d-4a27-8d93-508db41dcc81 None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:13.081008 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6068ca-5aa2-4ecf-9a32-1ce17316251b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:13.082509 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6068ca-5aa2-4ecf-9a32-1ce17316251b None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:13.083073 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6068ca-5aa2-4ecf-9a32-1ce17316251b None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:13.083513 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6068ca-5aa2-4ecf-9a32-1ce17316251b None admin] PATH_INFO: `/v3/users` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:13.084770 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6068ca-5aa2-4ecf-9a32-1ce17316251b None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 11:58:13.085262 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6068ca-5aa2-4ecf-9a32-1ce17316251b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:13.087928 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6068ca-5aa2-4ecf-9a32-1ce17316251b None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:13.116134 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 298/591] 10.222.0.23 () {56 vars in 1137 bytes} [Thu May 19 11:58:12 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:58:13.117327 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6068ca-5aa2-4ecf-9a32-1ce17316251b None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:13.121093 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fdfcd4-eb3b-465b-9b91-af014ec373b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:13.122944 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fdfcd4-eb3b-465b-9b91-af014ec373b7 None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:13.124295 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fdfcd4-eb3b-465b-9b91-af014ec373b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:13.124656 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fdfcd4-eb3b-465b-9b91-af014ec373b7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:13.138701 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2fdfcd4-eb3b-465b-9b91-af014ec373b7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:13.141552 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2fdfcd4-eb3b-465b-9b91-af014ec373b7 None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:13.164155 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application [None req-b2fdfcd4-eb3b-465b-9b91-af014ec373b7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 11:58:13.164155 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 11:58:13.164155 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 11:58:13.164155 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 11:58:13.164155 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 11:58:13.164155 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 11:58:13.164155 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 11:58:13.164155 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 11:58:13.164155 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 11:58:13.164155 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 11:58:13.164155 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 11:58:13.164155 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 11:58:13.164155 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 11:58:13.164155 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 11:58:13.164155 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 11:58:13.165096 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 11:58:13.165096 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 11:58:13.165096 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 11:58:13.165096 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 11:58:13.165096 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 11:58:13.165096 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 11:58:13.165096 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.get_or_create( May 19 11:58:13.165096 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 11:58:13.165096 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application with Lock( May 19 11:58:13.165096 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 11:58:13.165096 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._enter() May 19 11:58:13.165096 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 11:58:13.165096 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 11:58:13.165096 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 11:58:13.165096 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.creator() May 19 11:58:13.165096 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 11:58:13.165096 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application created_value = creator( May 19 11:58:13.165969 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 11:58:13.165969 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 11:58:13.165969 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 11:58:13.165969 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application May 19 11:58:13.165969 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 294/592] 10.222.0.23 () {56 vars in 1108 bytes} [Thu May 19 11:58:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 11:58:13.166706 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fdfcd4-eb3b-465b-9b91-af014ec373b7 None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:13.245990 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6068ca-5aa2-4ecf-9a32-1ce17316251b None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:13.251656 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:58:13.297861 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fdfcd4-eb3b-465b-9b91-af014ec373b7 None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:13.298399 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b78254-ae73-46ac-803c-83d875f25805 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:13.298907 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b78254-ae73-46ac-803c-83d875f25805 None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:13.299069 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b78254-ae73-46ac-803c-83d875f25805 None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:13.299341 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b78254-ae73-46ac-803c-83d875f25805 None admin] PATH_INFO: `/v3/projects` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:13.300194 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b78254-ae73-46ac-803c-83d875f25805 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 11:58:13.300505 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b78254-ae73-46ac-803c-83d875f25805 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:13.302989 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b78254-ae73-46ac-803c-83d875f25805 None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:13.308653 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:58:13.323879 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 299/593] 10.222.0.23 () {56 vars in 1148 bytes} [Thu May 19 11:58:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:58:13.323879 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b78254-ae73-46ac-803c-83d875f25805 None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:13.345960 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f42126-11df-483c-a2a7-8fa326c99657 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:13.347097 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f42126-11df-483c-a2a7-8fa326c99657 None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:13.347388 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f42126-11df-483c-a2a7-8fa326c99657 None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:13.347670 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f42126-11df-483c-a2a7-8fa326c99657 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:13.362588 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f42126-11df-483c-a2a7-8fa326c99657 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:13.364089 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f42126-11df-483c-a2a7-8fa326c99657 None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:13.387350 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application [None req-66f42126-11df-483c-a2a7-8fa326c99657 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 11:58:13.387350 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 11:58:13.387350 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 11:58:13.387350 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 11:58:13.387350 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 11:58:13.387350 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 11:58:13.387350 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 11:58:13.387350 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 11:58:13.387350 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 11:58:13.387350 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 11:58:13.387350 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 11:58:13.387350 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 11:58:13.387350 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 11:58:13.387350 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 11:58:13.387350 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 11:58:13.388337 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 11:58:13.388337 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 11:58:13.388337 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 11:58:13.388337 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 11:58:13.388337 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 11:58:13.388337 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 11:58:13.388337 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.get_or_create( May 19 11:58:13.388337 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 11:58:13.388337 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application with Lock( May 19 11:58:13.388337 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 11:58:13.388337 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._enter() May 19 11:58:13.388337 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 11:58:13.388337 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 11:58:13.388337 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 11:58:13.388337 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.creator() May 19 11:58:13.388337 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 11:58:13.388337 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application created_value = creator( May 19 11:58:13.389798 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 11:58:13.389798 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 11:58:13.389798 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 11:58:13.389798 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application May 19 11:58:13.389798 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 295/594] 10.222.0.23 () {56 vars in 1108 bytes} [Thu May 19 11:58:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 11:58:13.390522 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f42126-11df-483c-a2a7-8fa326c99657 None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:13.444988 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b78254-ae73-46ac-803c-83d875f25805 None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:13.499908 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920623e7-8559-4e74-987c-56850f86ac68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:13.501112 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920623e7-8559-4e74-987c-56850f86ac68 None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:13.501406 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920623e7-8559-4e74-987c-56850f86ac68 None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:13.501689 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920623e7-8559-4e74-987c-56850f86ac68 None admin] PATH_INFO: `/v3/domains` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:13.502499 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920623e7-8559-4e74-987c-56850f86ac68 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 11:58:13.502817 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920623e7-8559-4e74-987c-56850f86ac68 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:13.504338 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920623e7-8559-4e74-987c-56850f86ac68 None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:13.518947 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f42126-11df-483c-a2a7-8fa326c99657 None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:13.522736 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 300/595] 10.222.0.23 () {56 vars in 1109 bytes} [Thu May 19 11:58:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:58:13.523741 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920623e7-8559-4e74-987c-56850f86ac68 None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:13.558404 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d95948-a27d-40e0-9e95-62a38f290926 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:13.559146 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:58:13.560293 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d95948-a27d-40e0-9e95-62a38f290926 None admin] REQUEST_METHOD: `PUT` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:13.561409 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d95948-a27d-40e0-9e95-62a38f290926 None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:13.561900 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d95948-a27d-40e0-9e95-62a38f290926 None admin] PATH_INFO: `/v3/projects/d70d25bf69314820b8aaecb65576978a/users/8800254c0e784e42a791fb9abc274d79/roles/7c59061ba7064a0db6032433fb0eec6b` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:13.578748 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d95948-a27d-40e0-9e95-62a38f290926 None admin] RBAC: Authorizing `identity:create_grant(project_id=d70d25bf69314820b8aaecb65576978a, user_id=8800254c0e784e42a791fb9abc274d79, role_id=7c59061ba7064a0db6032433fb0eec6b)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:13.580644 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d95948-a27d-40e0-9e95-62a38f290926 None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:13.646288 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920623e7-8559-4e74-987c-56850f86ac68 None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:13.656084 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 296/596] 10.222.0.23 () {58 vars in 1439 bytes} [Thu May 19 11:58:13 2022] PUT /identity/v3/projects/d70d25bf69314820b8aaecb65576978a/users/8800254c0e784e42a791fb9abc274d79/roles/7c59061ba7064a0db6032433fb0eec6b => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 11:58:13.657546 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d95948-a27d-40e0-9e95-62a38f290926 None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:13.687701 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd587478-fc9f-4588-84e1-c5a7500daa35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:13.688464 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd587478-fc9f-4588-84e1-c5a7500daa35 None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:13.688758 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd587478-fc9f-4588-84e1-c5a7500daa35 None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:13.689092 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd587478-fc9f-4588-84e1-c5a7500daa35 None admin] PATH_INFO: `/v3/domains` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:13.689850 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd587478-fc9f-4588-84e1-c5a7500daa35 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 11:58:13.690602 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:58:13.691125 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd587478-fc9f-4588-84e1-c5a7500daa35 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:13.692727 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd587478-fc9f-4588-84e1-c5a7500daa35 None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:13.707458 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 301/597] 10.222.0.23 () {56 vars in 1109 bytes} [Thu May 19 11:58:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:58:13.712683 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd587478-fc9f-4588-84e1-c5a7500daa35 None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:13.923142 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d95948-a27d-40e0-9e95-62a38f290926 None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:13.963635 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75166d5-3bb7-457e-8c44-87b4ffa1984e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:13.964671 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75166d5-3bb7-457e-8c44-87b4ffa1984e None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:13.964962 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75166d5-3bb7-457e-8c44-87b4ffa1984e None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:13.965244 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75166d5-3bb7-457e-8c44-87b4ffa1984e None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:13.971175 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:58:13.974342 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75166d5-3bb7-457e-8c44-87b4ffa1984e None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:13.975817 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75166d5-3bb7-457e-8c44-87b4ffa1984e None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:13.989333 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd587478-fc9f-4588-84e1-c5a7500daa35 None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:13.991648 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application [None req-c75166d5-3bb7-457e-8c44-87b4ffa1984e None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. May 19 11:58:13.991648 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 11:58:13.991648 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 11:58:13.991648 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 11:58:13.991648 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 11:58:13.991648 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 11:58:13.991648 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 11:58:13.991648 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 11:58:13.991648 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 11:58:13.991648 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 11:58:13.991648 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 11:58:13.991648 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 11:58:13.991648 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 11:58:13.991648 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 11:58:13.991648 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 11:58:13.991648 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 11:58:13.992613 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 11:58:13.992613 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 11:58:13.992613 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 11:58:13.992613 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 11:58:13.992613 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 11:58:13.992613 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 11:58:13.992613 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 11:58:13.992613 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 11:58:13.992613 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 11:58:13.992613 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.get_or_create( May 19 11:58:13.992613 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 11:58:13.992613 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application with Lock( May 19 11:58:13.992613 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 11:58:13.992613 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._enter() May 19 11:58:13.992613 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 11:58:13.992613 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 11:58:13.992613 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 11:58:13.992613 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.creator() May 19 11:58:13.993953 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 11:58:13.993953 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application created_value = creator( May 19 11:58:13.993953 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 11:58:13.993953 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 11:58:13.993953 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 11:58:13.993953 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 11:58:13.993953 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 11:58:13.993953 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 11:58:13.993953 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. May 19 11:58:13.993953 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application May 19 11:58:13.993953 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 297/598] 10.222.0.23 () {56 vars in 1102 bytes} [Thu May 19 11:58:13 2022] GET /identity/v3/users/neutron => generated 85 bytes in 336 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 11:58:14.001076 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75166d5-3bb7-457e-8c44-87b4ffa1984e None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:14.028895 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2900ac55-07f5-4caf-ac4d-b8d8eae2e7d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:14.030394 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2900ac55-07f5-4caf-ac4d-b8d8eae2e7d1 None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:14.030766 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2900ac55-07f5-4caf-ac4d-b8d8eae2e7d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:14.031045 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2900ac55-07f5-4caf-ac4d-b8d8eae2e7d1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:14.051064 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2900ac55-07f5-4caf-ac4d-b8d8eae2e7d1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:14.052609 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2900ac55-07f5-4caf-ac4d-b8d8eae2e7d1 None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:14.075445 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application [None req-2900ac55-07f5-4caf-ac4d-b8d8eae2e7d1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 11:58:14.075445 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 11:58:14.075445 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 11:58:14.075445 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 11:58:14.075445 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 11:58:14.075445 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 11:58:14.075445 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 11:58:14.075445 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 11:58:14.075445 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 11:58:14.075445 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 11:58:14.075445 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 11:58:14.075445 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 11:58:14.075445 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 11:58:14.075445 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 11:58:14.075445 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 11:58:14.078350 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 11:58:14.078350 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 11:58:14.078350 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 11:58:14.078350 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 11:58:14.078350 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 11:58:14.078350 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 11:58:14.078350 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.get_or_create( May 19 11:58:14.078350 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 11:58:14.078350 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application with Lock( May 19 11:58:14.078350 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 11:58:14.078350 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._enter() May 19 11:58:14.078350 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 11:58:14.078350 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 11:58:14.078350 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 11:58:14.078350 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.creator() May 19 11:58:14.078350 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 11:58:14.078350 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application created_value = creator( May 19 11:58:14.079853 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 11:58:14.079853 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 11:58:14.079853 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 11:58:14.079853 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 11:58:14.079853 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 11:58:14.079853 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 11:58:14.079853 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 11:58:14.079853 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application May 19 11:58:14.079853 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 302/599] 10.222.0.23 () {56 vars in 1111 bytes} [Thu May 19 11:58:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 366 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 11:58:14.087833 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2900ac55-07f5-4caf-ac4d-b8d8eae2e7d1 None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:14.193552 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:58:14.203106 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2900ac55-07f5-4caf-ac4d-b8d8eae2e7d1 None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:14.244019 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd4bbcf-4325-4dfb-9e32-f212bf66dac0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:14.244697 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd4bbcf-4325-4dfb-9e32-f212bf66dac0 None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:14.244982 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd4bbcf-4325-4dfb-9e32-f212bf66dac0 None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:14.245256 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd4bbcf-4325-4dfb-9e32-f212bf66dac0 None admin] PATH_INFO: `/v3/projects` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:14.246049 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd4bbcf-4325-4dfb-9e32-f212bf66dac0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 11:58:14.246356 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd4bbcf-4325-4dfb-9e32-f212bf66dac0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:14.247833 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd4bbcf-4325-4dfb-9e32-f212bf66dac0 None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:14.263075 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 303/600] 10.222.0.23 () {56 vars in 1148 bytes} [Thu May 19 11:58:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:58:14.271543 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd4bbcf-4325-4dfb-9e32-f212bf66dac0 None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:14.278247 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:58:14.299141 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75166d5-3bb7-457e-8c44-87b4ffa1984e None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:14.365770 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116d523a-ca8d-48d8-b4be-d226c378dc70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:14.366463 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116d523a-ca8d-48d8-b4be-d226c378dc70 None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:14.366745 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116d523a-ca8d-48d8-b4be-d226c378dc70 None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:14.367021 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116d523a-ca8d-48d8-b4be-d226c378dc70 None admin] PATH_INFO: `/v3/users` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:14.367916 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116d523a-ca8d-48d8-b4be-d226c378dc70 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 11:58:14.368230 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116d523a-ca8d-48d8-b4be-d226c378dc70 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:14.369719 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116d523a-ca8d-48d8-b4be-d226c378dc70 None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:14.411800 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 298/601] 10.222.0.23 () {56 vars in 1139 bytes} [Thu May 19 11:58:14 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 409 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:58:14.416892 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116d523a-ca8d-48d8-b4be-d226c378dc70 None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:14.456599 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd4bbcf-4325-4dfb-9e32-f212bf66dac0 None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:14.495523 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90b51ad-a34c-4f3c-9ec2-85386854886b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:14.496444 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90b51ad-a34c-4f3c-9ec2-85386854886b None admin] REQUEST_METHOD: `PUT` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:14.496733 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90b51ad-a34c-4f3c-9ec2-85386854886b None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:14.497011 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90b51ad-a34c-4f3c-9ec2-85386854886b None admin] PATH_INFO: `/v3/projects/d70d25bf69314820b8aaecb65576978a/users/21697802f2be42f9baa8e07490d5ebdd/roles/7c59061ba7064a0db6032433fb0eec6b` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:14.528600 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90b51ad-a34c-4f3c-9ec2-85386854886b None admin] RBAC: Authorizing `identity:create_grant(project_id=d70d25bf69314820b8aaecb65576978a, user_id=21697802f2be42f9baa8e07490d5ebdd, role_id=7c59061ba7064a0db6032433fb0eec6b)` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:14.530657 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90b51ad-a34c-4f3c-9ec2-85386854886b None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:14.579612 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116d523a-ca8d-48d8-b4be-d226c378dc70 None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:14.611163 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:58:14.619997 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a17c6b3-3f95-44b4-a083-786c5c31a258 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:14.620808 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a17c6b3-3f95-44b4-a083-786c5c31a258 None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:14.621091 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a17c6b3-3f95-44b4-a083-786c5c31a258 None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:14.621367 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a17c6b3-3f95-44b4-a083-786c5c31a258 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:14.631709 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a17c6b3-3f95-44b4-a083-786c5c31a258 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:14.633205 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a17c6b3-3f95-44b4-a083-786c5c31a258 None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:14.650818 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 304/602] 10.222.0.23 () {58 vars in 1439 bytes} [Thu May 19 11:58:14 2022] PUT /identity/v3/projects/d70d25bf69314820b8aaecb65576978a/users/21697802f2be42f9baa8e07490d5ebdd/roles/7c59061ba7064a0db6032433fb0eec6b => generated 0 bytes in 380 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 11:58:14.652548 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application [None req-5a17c6b3-3f95-44b4-a083-786c5c31a258 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 11:58:14.652548 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 11:58:14.652548 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 11:58:14.652548 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 11:58:14.652548 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 11:58:14.652548 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 11:58:14.652548 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 11:58:14.652548 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 11:58:14.652548 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 11:58:14.652548 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 11:58:14.652548 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 11:58:14.652548 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 11:58:14.652548 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 11:58:14.652548 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 11:58:14.652548 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 11:58:14.661689 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 11:58:14.661689 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 11:58:14.661689 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 11:58:14.661689 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 11:58:14.661689 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 11:58:14.661689 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 11:58:14.661689 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.get_or_create( May 19 11:58:14.661689 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 11:58:14.661689 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application with Lock( May 19 11:58:14.661689 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 11:58:14.661689 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._enter() May 19 11:58:14.661689 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 11:58:14.661689 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 11:58:14.661689 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 11:58:14.661689 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.creator() May 19 11:58:14.661689 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 11:58:14.661689 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application created_value = creator( May 19 11:58:14.662576 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 11:58:14.662576 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 11:58:14.662576 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 11:58:14.662576 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application May 19 11:58:14.662801 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c7320f-8be2-4476-a4de-e04a2b35da72 None None] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:14.662801 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c7320f-8be2-4476-a4de-e04a2b35da72 None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:14.662801 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c7320f-8be2-4476-a4de-e04a2b35da72 None None] PATH_INFO: `/` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:14.662994 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 299/603] 10.222.0.23 () {56 vars in 1108 bytes} [Thu May 19 11:58:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 11:58:14.663852 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 305/604] 10.222.0.23 () {54 vars in 926 bytes} [Thu May 19 11:58:14 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 11:58:14.666255 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a17c6b3-3f95-44b4-a083-786c5c31a258 None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:14.675062 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1db4db-cdb3-4905-a663-9bd683eb3dcf None None] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:14.675402 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1db4db-cdb3-4905-a663-9bd683eb3dcf None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:14.675687 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1db4db-cdb3-4905-a663-9bd683eb3dcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:14.711178 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.auth.core [None req-ec1db4db-cdb3-4905-a663-9bd683eb3dcf None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 11:58:14.713873 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-ec1db4db-cdb3-4905-a663-9bd683eb3dcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 11:58:14.834225 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:58:14.872414 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a17c6b3-3f95-44b4-a083-786c5c31a258 None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:14.877512 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 306/605] 10.222.0.23 () {58 vars in 1023 bytes} [Thu May 19 11:58:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 11:58:14.910517 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1db4db-cdb3-4905-a663-9bd683eb3dcf None None] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:14.916957 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2bfd16-0c81-44a4-8ced-03c4af5042aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:14.917956 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2bfd16-0c81-44a4-8ced-03c4af5042aa None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:14.918382 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2bfd16-0c81-44a4-8ced-03c4af5042aa None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:14.918799 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2bfd16-0c81-44a4-8ced-03c4af5042aa None admin] PATH_INFO: `/v3/domains` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:14.919671 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2bfd16-0c81-44a4-8ced-03c4af5042aa None admin] RBAC: Adding query filter params (name=Default) {{(pid=80539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 11:58:14.920677 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2bfd16-0c81-44a4-8ced-03c4af5042aa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:14.922526 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2bfd16-0c81-44a4-8ced-03c4af5042aa None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:14.938083 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 300/606] 10.222.0.23 () {56 vars in 1109 bytes} [Thu May 19 11:58:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:58:14.943819 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2bfd16-0c81-44a4-8ced-03c4af5042aa None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:15.008834 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:58:15.054887 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1db4db-cdb3-4905-a663-9bd683eb3dcf None None] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:15.102524 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316e74c2-47c1-4d27-987c-18abd97ec4b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:15.103408 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316e74c2-47c1-4d27-987c-18abd97ec4b7 None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:15.103693 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316e74c2-47c1-4d27-987c-18abd97ec4b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:15.104018 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316e74c2-47c1-4d27-987c-18abd97ec4b7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:15.129908 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316e74c2-47c1-4d27-987c-18abd97ec4b7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 11:58:15.142409 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application [None req-316e74c2-47c1-4d27-987c-18abd97ec4b7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 11:58:15.142409 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 11:58:15.142409 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 11:58:15.142409 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 11:58:15.142409 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 11:58:15.142409 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 11:58:15.142409 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 11:58:15.142409 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 11:58:15.142409 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 11:58:15.142409 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 11:58:15.142409 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 11:58:15.142409 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 11:58:15.142409 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 11:58:15.142409 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 11:58:15.142409 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 11:58:15.142409 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 11:58:15.143405 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 11:58:15.143405 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 11:58:15.143405 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 11:58:15.143405 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 11:58:15.143405 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 11:58:15.143405 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 11:58:15.143405 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 11:58:15.143405 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 11:58:15.143405 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 11:58:15.143405 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.get_or_create( May 19 11:58:15.143405 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 11:58:15.143405 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application with Lock( May 19 11:58:15.143405 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 11:58:15.143405 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._enter() May 19 11:58:15.143405 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 11:58:15.143405 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 11:58:15.144277 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 11:58:15.144277 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.creator() May 19 11:58:15.144277 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 11:58:15.144277 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application created_value = creator( May 19 11:58:15.144277 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 11:58:15.144277 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 11:58:15.144277 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 11:58:15.144277 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 11:58:15.144277 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 11:58:15.144277 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 11:58:15.144277 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 11:58:15.144277 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application May 19 11:58:15.145489 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 307/607] 10.222.0.23 () {56 vars in 1102 bytes} [Thu May 19 11:58:14 2022] GET /identity/v3/roles/service => generated 85 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 11:58:15.150219 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316e74c2-47c1-4d27-987c-18abd97ec4b7 None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:15.163515 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2bfd16-0c81-44a4-8ced-03c4af5042aa None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:15.240813 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78de14c6-6653-4e6f-928e-7e7eccb78872 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:15.241997 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78de14c6-6653-4e6f-928e-7e7eccb78872 None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:15.242512 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78de14c6-6653-4e6f-928e-7e7eccb78872 None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:15.242993 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78de14c6-6653-4e6f-928e-7e7eccb78872 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:15.248709 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:58:15.257283 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78de14c6-6653-4e6f-928e-7e7eccb78872 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:15.258758 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78de14c6-6653-4e6f-928e-7e7eccb78872 None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:15.281695 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application [None req-78de14c6-6653-4e6f-928e-7e7eccb78872 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 11:58:15.281695 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 11:58:15.281695 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 11:58:15.281695 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 11:58:15.281695 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 11:58:15.281695 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 11:58:15.281695 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 11:58:15.281695 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 11:58:15.281695 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 11:58:15.281695 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 11:58:15.281695 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 11:58:15.281695 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 11:58:15.281695 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 11:58:15.281695 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 11:58:15.281695 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 11:58:15.282685 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 11:58:15.282685 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 11:58:15.282685 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 11:58:15.282685 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 11:58:15.282685 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 11:58:15.282685 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 11:58:15.282685 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.get_or_create( May 19 11:58:15.282685 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 11:58:15.282685 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application with Lock( May 19 11:58:15.282685 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 11:58:15.282685 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._enter() May 19 11:58:15.282685 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 11:58:15.282685 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 11:58:15.282685 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 11:58:15.282685 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.creator() May 19 11:58:15.282685 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 11:58:15.282685 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application created_value = creator( May 19 11:58:15.283586 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 11:58:15.283586 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 11:58:15.283586 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 11:58:15.283586 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 11:58:15.283586 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 11:58:15.283586 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 11:58:15.283586 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 11:58:15.283586 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application May 19 11:58:15.286661 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 301/608] 10.222.0.23 () {56 vars in 1111 bytes} [Thu May 19 11:58:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 340 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 11:58:15.288994 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78de14c6-6653-4e6f-928e-7e7eccb78872 None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:15.350032 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316e74c2-47c1-4d27-987c-18abd97ec4b7 None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:15.440965 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ccfac1-d0cb-4b01-a6ba-ef08b1ed7bd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:15.441654 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ccfac1-d0cb-4b01-a6ba-ef08b1ed7bd4 None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:15.441946 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ccfac1-d0cb-4b01-a6ba-ef08b1ed7bd4 None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:15.442229 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ccfac1-d0cb-4b01-a6ba-ef08b1ed7bd4 None admin] PATH_INFO: `/v3/roles` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:15.442996 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ccfac1-d0cb-4b01-a6ba-ef08b1ed7bd4 None admin] RBAC: Adding query filter params (name=service) {{(pid=80540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 11:58:15.443313 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ccfac1-d0cb-4b01-a6ba-ef08b1ed7bd4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:15.444741 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ccfac1-d0cb-4b01-a6ba-ef08b1ed7bd4 None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:15.464416 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 308/609] 10.222.0.23 () {56 vars in 1103 bytes} [Thu May 19 11:58:15 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 11:58:15.472176 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ccfac1-d0cb-4b01-a6ba-ef08b1ed7bd4 None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:15.476856 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:58:15.485820 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78de14c6-6653-4e6f-928e-7e7eccb78872 None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:15.558001 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6e4d3e-ad2d-4b2c-89b1-b403c83aece3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:15.558680 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6e4d3e-ad2d-4b2c-89b1-b403c83aece3 None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:15.558966 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6e4d3e-ad2d-4b2c-89b1-b403c83aece3 None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:15.559241 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6e4d3e-ad2d-4b2c-89b1-b403c83aece3 None admin] PATH_INFO: `/v3/projects` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:15.560070 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6e4d3e-ad2d-4b2c-89b1-b403c83aece3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 11:58:15.560384 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6e4d3e-ad2d-4b2c-89b1-b403c83aece3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:15.562673 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6e4d3e-ad2d-4b2c-89b1-b403c83aece3 None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:15.591509 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 302/610] 10.222.0.23 () {56 vars in 1148 bytes} [Thu May 19 11:58:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 11:58:15.595559 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6e4d3e-ad2d-4b2c-89b1-b403c83aece3 None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:15.660115 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ccfac1-d0cb-4b01-a6ba-ef08b1ed7bd4 None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:15.677255 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:58:15.716360 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e64ccb-a211-43ee-bba8-e4384c94b4ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:15.717117 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6e4d3e-ad2d-4b2c-89b1-b403c83aece3 None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:15.717523 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e64ccb-a211-43ee-bba8-e4384c94b4ba None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:15.717945 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e64ccb-a211-43ee-bba8-e4384c94b4ba None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:15.718352 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e64ccb-a211-43ee-bba8-e4384c94b4ba None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:15.734372 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e64ccb-a211-43ee-bba8-e4384c94b4ba None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:15.736447 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e64ccb-a211-43ee-bba8-e4384c94b4ba None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:15.760348 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b167282d-60b9-418a-bbb3-03ea0331dfa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:15.761242 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b167282d-60b9-418a-bbb3-03ea0331dfa7 None admin] REQUEST_METHOD: `PUT` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:15.761527 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b167282d-60b9-418a-bbb3-03ea0331dfa7 None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:15.761805 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b167282d-60b9-418a-bbb3-03ea0331dfa7 None admin] PATH_INFO: `/v3/projects/d70d25bf69314820b8aaecb65576978a/users/1a00b3fa7497456895e5e0bf105c4fc6/roles/7c59061ba7064a0db6032433fb0eec6b` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:15.763352 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application [None req-a3e64ccb-a211-43ee-bba8-e4384c94b4ba None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 11:58:15.763352 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 11:58:15.763352 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 11:58:15.763352 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 11:58:15.763352 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 11:58:15.763352 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 11:58:15.763352 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 11:58:15.763352 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 11:58:15.763352 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 11:58:15.763352 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 11:58:15.763352 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 11:58:15.763352 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 11:58:15.763352 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 11:58:15.763352 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 11:58:15.763352 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 11:58:15.764975 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 11:58:15.764975 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 11:58:15.764975 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 11:58:15.764975 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 11:58:15.764975 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 11:58:15.764975 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 11:58:15.764975 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.get_or_create( May 19 11:58:15.764975 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 11:58:15.764975 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application with Lock( May 19 11:58:15.764975 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 11:58:15.764975 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._enter() May 19 11:58:15.764975 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 11:58:15.764975 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 11:58:15.764975 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 11:58:15.764975 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.creator() May 19 11:58:15.764975 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 11:58:15.764975 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application created_value = creator( May 19 11:58:15.765842 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 11:58:15.765842 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 11:58:15.765842 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 11:58:15.765842 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application May 19 11:58:15.773417 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 309/611] 10.222.0.23 () {56 vars in 1108 bytes} [Thu May 19 11:58:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 297 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 11:58:15.773417 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b66b4f-949f-482f-99b9-70481e7f9ed4 None None] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:15.773417 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b66b4f-949f-482f-99b9-70481e7f9ed4 None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:15.773417 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b66b4f-949f-482f-99b9-70481e7f9ed4 None None] PATH_INFO: `/` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:15.776118 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 310/612] 10.222.0.23 () {54 vars in 926 bytes} [Thu May 19 11:58:15 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 11:58:15.777314 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b66b4f-949f-482f-99b9-70481e7f9ed4 None None] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:15.788569 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b167282d-60b9-418a-bbb3-03ea0331dfa7 None admin] RBAC: Authorizing `identity:create_grant(project_id=d70d25bf69314820b8aaecb65576978a, user_id=1a00b3fa7497456895e5e0bf105c4fc6, role_id=7c59061ba7064a0db6032433fb0eec6b)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:15.790031 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b167282d-60b9-418a-bbb3-03ea0331dfa7 None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:15.852041 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 303/613] 10.222.0.23 () {58 vars in 1439 bytes} [Thu May 19 11:58:15 2022] PUT /identity/v3/projects/d70d25bf69314820b8aaecb65576978a/users/1a00b3fa7497456895e5e0bf105c4fc6/roles/7c59061ba7064a0db6032433fb0eec6b => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 11:58:15.854849 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60631592-9abf-4474-a253-04be710633dc None None] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:15.855804 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60631592-9abf-4474-a253-04be710633dc None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:15.857202 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60631592-9abf-4474-a253-04be710633dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:15.895431 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.auth.core [None req-60631592-9abf-4474-a253-04be710633dc None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 11:58:15.899507 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-60631592-9abf-4474-a253-04be710633dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 11:58:15.907376 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:58:15.956459 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b66b4f-949f-482f-99b9-70481e7f9ed4 None None] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:16.034868 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a985839-a52d-4d64-b289-0d833a615a33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:16.036823 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a985839-a52d-4d64-b289-0d833a615a33 None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:16.037192 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a985839-a52d-4d64-b289-0d833a615a33 None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:16.037563 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a985839-a52d-4d64-b289-0d833a615a33 None admin] PATH_INFO: `/v3/domains` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:16.039290 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a985839-a52d-4d64-b289-0d833a615a33 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 11:58:16.039908 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a985839-a52d-4d64-b289-0d833a615a33 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:16.043067 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a985839-a52d-4d64-b289-0d833a615a33 None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:16.059453 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 304/614] 10.222.0.23 () {58 vars in 1023 bytes} [Thu May 19 11:58:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 11:58:16.070746 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 311/615] 10.222.0.23 () {56 vars in 1109 bytes} [Thu May 19 11:58:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:58:16.081209 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60631592-9abf-4474-a253-04be710633dc None None] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:16.106164 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a985839-a52d-4d64-b289-0d833a615a33 None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:16.135010 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:58:16.261072 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60631592-9abf-4474-a253-04be710633dc None None] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:16.284569 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a985839-a52d-4d64-b289-0d833a615a33 None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:16.313329 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc16f7e-2ec2-4601-9f65-401fa3075a35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:16.314298 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc16f7e-2ec2-4601-9f65-401fa3075a35 None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:16.314587 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc16f7e-2ec2-4601-9f65-401fa3075a35 None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:16.314864 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc16f7e-2ec2-4601-9f65-401fa3075a35 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:16.323260 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc16f7e-2ec2-4601-9f65-401fa3075a35 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:16.324833 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc16f7e-2ec2-4601-9f65-401fa3075a35 None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:16.325698 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ecf3431-5a81-4547-b243-403fc349c639 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:16.327018 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecf3431-5a81-4547-b243-403fc349c639 None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:16.327342 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecf3431-5a81-4547-b243-403fc349c639 None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:16.327612 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecf3431-5a81-4547-b243-403fc349c639 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:16.338803 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ecf3431-5a81-4547-b243-403fc349c639 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 11:58:16.340657 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application [None req-4cc16f7e-2ec2-4601-9f65-401fa3075a35 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 19 11:58:16.340657 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 11:58:16.340657 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 11:58:16.340657 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 11:58:16.340657 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 11:58:16.340657 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 11:58:16.340657 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 11:58:16.340657 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 11:58:16.340657 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 11:58:16.340657 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 11:58:16.340657 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 11:58:16.340657 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 11:58:16.340657 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 11:58:16.340657 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 11:58:16.340657 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 11:58:16.340657 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 11:58:16.341562 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 11:58:16.341562 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 11:58:16.341562 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 11:58:16.341562 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 11:58:16.341562 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 11:58:16.341562 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 11:58:16.341562 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 11:58:16.341562 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 11:58:16.341562 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 11:58:16.341562 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.get_or_create( May 19 11:58:16.341562 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 11:58:16.341562 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application with Lock( May 19 11:58:16.341562 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 11:58:16.341562 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._enter() May 19 11:58:16.341562 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 11:58:16.341562 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 11:58:16.341562 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 11:58:16.341562 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.creator() May 19 11:58:16.343524 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 11:58:16.343524 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application created_value = creator( May 19 11:58:16.343524 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 11:58:16.343524 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 11:58:16.343524 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 11:58:16.343524 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 11:58:16.343524 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 11:58:16.343524 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 11:58:16.343524 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 19 11:58:16.343524 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application May 19 11:58:16.343524 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 305/616] 10.222.0.23 () {56 vars in 1096 bytes} [Thu May 19 11:58:16 2022] GET /identity/v3/users/swift => generated 83 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 11:58:16.344627 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56afb895-2db9-4429-941e-fd0e8b093d7c None None] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:16.344795 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56afb895-2db9-4429-941e-fd0e8b093d7c None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:16.344963 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56afb895-2db9-4429-941e-fd0e8b093d7c None None] PATH_INFO: `/` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:16.347729 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application [None req-8ecf3431-5a81-4547-b243-403fc349c639 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 11:58:16.347729 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 11:58:16.347729 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 11:58:16.347729 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 11:58:16.347729 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 11:58:16.347729 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 11:58:16.347729 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 11:58:16.347729 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 11:58:16.347729 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 11:58:16.347729 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 11:58:16.347729 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 11:58:16.347729 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 11:58:16.347729 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 11:58:16.347729 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 11:58:16.347729 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 11:58:16.347729 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 11:58:16.350208 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 11:58:16.350208 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 11:58:16.350208 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 11:58:16.350208 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 11:58:16.350208 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 11:58:16.350208 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 11:58:16.350208 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 11:58:16.350208 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 11:58:16.350208 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 11:58:16.350208 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.get_or_create( May 19 11:58:16.350208 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 11:58:16.350208 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application with Lock( May 19 11:58:16.350208 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 11:58:16.350208 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._enter() May 19 11:58:16.350208 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 11:58:16.350208 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 11:58:16.352155 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 11:58:16.352155 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.creator() May 19 11:58:16.352155 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 11:58:16.352155 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application created_value = creator( May 19 11:58:16.352155 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 11:58:16.352155 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 11:58:16.352155 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 11:58:16.352155 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 11:58:16.352155 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 11:58:16.352155 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 11:58:16.352155 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 11:58:16.352155 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application May 19 11:58:16.352155 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 312/618] 10.222.0.23 () {56 vars in 1102 bytes} [Thu May 19 11:58:16 2022] GET /identity/v3/roles/service => generated 85 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 11:58:16.353736 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 306/618] 10.222.0.23 () {54 vars in 926 bytes} [Thu May 19 11:58:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 11:58:16.354833 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:58:16.355003 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ecf3431-5a81-4547-b243-403fc349c639 None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:16.356507 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81541389-c5b2-47f8-82cf-840455277865 None None] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:16.356963 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81541389-c5b2-47f8-82cf-840455277865 None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:16.357390 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81541389-c5b2-47f8-82cf-840455277865 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:16.381309 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.auth.core [None req-81541389-c5b2-47f8-82cf-840455277865 None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 11:58:16.383111 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-81541389-c5b2-47f8-82cf-840455277865 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 11:58:16.449872 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:58:16.473727 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ecf3431-5a81-4547-b243-403fc349c639 None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:16.476047 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 307/619] 10.222.0.23 () {58 vars in 1023 bytes} [Thu May 19 11:58:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 11:58:16.477161 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81541389-c5b2-47f8-82cf-840455277865 None None] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:16.527579 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687539f3-f1c3-4c51-ba0e-b0f760146cb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:16.528380 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687539f3-f1c3-4c51-ba0e-b0f760146cb4 None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:16.528762 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687539f3-f1c3-4c51-ba0e-b0f760146cb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:16.529046 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687539f3-f1c3-4c51-ba0e-b0f760146cb4 None admin] PATH_INFO: `/v3/users` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:16.530464 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687539f3-f1c3-4c51-ba0e-b0f760146cb4 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 11:58:16.530777 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687539f3-f1c3-4c51-ba0e-b0f760146cb4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:16.532375 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687539f3-f1c3-4c51-ba0e-b0f760146cb4 None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:16.552637 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 313/620] 10.222.0.23 () {56 vars in 1135 bytes} [Thu May 19 11:58:16 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:58:16.553665 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687539f3-f1c3-4c51-ba0e-b0f760146cb4 None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:16.585544 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81541389-c5b2-47f8-82cf-840455277865 None None] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:16.614370 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:58:16.622231 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2e4ad8-82e2-4a66-87fb-3c5b4a9d09ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:16.622906 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2e4ad8-82e2-4a66-87fb-3c5b4a9d09ad None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:16.623227 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2e4ad8-82e2-4a66-87fb-3c5b4a9d09ad None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:16.640170 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2e4ad8-82e2-4a66-87fb-3c5b4a9d09ad None admin] PATH_INFO: `/v3/roles` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:16.640170 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2e4ad8-82e2-4a66-87fb-3c5b4a9d09ad None admin] RBAC: Adding query filter params (name=service) {{(pid=80539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 11:58:16.640170 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2e4ad8-82e2-4a66-87fb-3c5b4a9d09ad None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:16.640170 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2e4ad8-82e2-4a66-87fb-3c5b4a9d09ad None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:16.640170 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 308/621] 10.222.0.23 () {56 vars in 1103 bytes} [Thu May 19 11:58:16 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:58:16.640170 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2e4ad8-82e2-4a66-87fb-3c5b4a9d09ad None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:16.665690 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687539f3-f1c3-4c51-ba0e-b0f760146cb4 None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:16.706493 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30c7789-bfb9-492c-804b-1f3437f86933 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:16.707396 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30c7789-bfb9-492c-804b-1f3437f86933 None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:16.707689 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30c7789-bfb9-492c-804b-1f3437f86933 None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:16.716913 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30c7789-bfb9-492c-804b-1f3437f86933 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:16.716913 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:58:16.720652 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30c7789-bfb9-492c-804b-1f3437f86933 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 11:58:16.728979 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application [None req-b30c7789-bfb9-492c-804b-1f3437f86933 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 11:58:16.728979 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 11:58:16.728979 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 11:58:16.728979 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 11:58:16.728979 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 11:58:16.728979 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 11:58:16.728979 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 11:58:16.728979 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 11:58:16.728979 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 11:58:16.728979 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 11:58:16.728979 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 11:58:16.728979 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 11:58:16.728979 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 11:58:16.728979 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 11:58:16.728979 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 11:58:16.728979 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 11:58:16.731113 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 11:58:16.731113 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 11:58:16.731113 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 11:58:16.731113 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 11:58:16.731113 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 11:58:16.731113 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 11:58:16.731113 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 11:58:16.731113 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 11:58:16.731113 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 11:58:16.731113 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.get_or_create( May 19 11:58:16.731113 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 11:58:16.731113 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application with Lock( May 19 11:58:16.731113 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 11:58:16.731113 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._enter() May 19 11:58:16.731113 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 11:58:16.731113 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 11:58:16.733651 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 11:58:16.733651 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.creator() May 19 11:58:16.733651 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 11:58:16.733651 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application created_value = creator( May 19 11:58:16.733651 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 11:58:16.733651 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 11:58:16.733651 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 11:58:16.733651 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 11:58:16.733651 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 11:58:16.733651 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 11:58:16.733651 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 11:58:16.733651 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application May 19 11:58:16.740109 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 314/622] 10.222.0.23 () {56 vars in 1102 bytes} [Thu May 19 11:58:16 2022] GET /identity/v3/roles/service => generated 85 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 11:58:16.740109 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30c7789-bfb9-492c-804b-1f3437f86933 None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:16.744674 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2e4ad8-82e2-4a66-87fb-3c5b4a9d09ad None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:16.785702 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e2efa7-9e9d-4d46-b0c7-4b35198c3771 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:16.786679 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e2efa7-9e9d-4d46-b0c7-4b35198c3771 None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:16.786960 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e2efa7-9e9d-4d46-b0c7-4b35198c3771 None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:16.787234 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e2efa7-9e9d-4d46-b0c7-4b35198c3771 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:16.799772 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e2efa7-9e9d-4d46-b0c7-4b35198c3771 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:16.801226 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e2efa7-9e9d-4d46-b0c7-4b35198c3771 None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:16.821753 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application [None req-c7e2efa7-9e9d-4d46-b0c7-4b35198c3771 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 11:58:16.821753 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 11:58:16.821753 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 11:58:16.821753 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 11:58:16.821753 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 11:58:16.821753 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 11:58:16.821753 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 11:58:16.821753 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 11:58:16.821753 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 11:58:16.821753 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 11:58:16.821753 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 11:58:16.821753 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 11:58:16.821753 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 11:58:16.821753 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 11:58:16.821753 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 11:58:16.822908 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 11:58:16.822908 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 11:58:16.822908 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 11:58:16.822908 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 11:58:16.822908 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 11:58:16.822908 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 11:58:16.822908 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.get_or_create( May 19 11:58:16.822908 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 11:58:16.822908 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application with Lock( May 19 11:58:16.822908 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 11:58:16.822908 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._enter() May 19 11:58:16.822908 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 11:58:16.822908 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 11:58:16.822908 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 11:58:16.822908 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.creator() May 19 11:58:16.822908 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 11:58:16.822908 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application created_value = creator( May 19 11:58:16.823983 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 11:58:16.823983 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 11:58:16.823983 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 11:58:16.823983 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application May 19 11:58:16.823983 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 309/623] 10.222.0.23 () {56 vars in 1108 bytes} [Thu May 19 11:58:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 11:58:16.824284 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e2efa7-9e9d-4d46-b0c7-4b35198c3771 None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:16.858891 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30c7789-bfb9-492c-804b-1f3437f86933 None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:16.874567 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:58:16.897168 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b980f45a-703e-4cc2-baac-3cd19fcc579c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:16.898243 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b980f45a-703e-4cc2-baac-3cd19fcc579c None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:16.898674 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b980f45a-703e-4cc2-baac-3cd19fcc579c None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:16.899093 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b980f45a-703e-4cc2-baac-3cd19fcc579c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:16.911227 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b980f45a-703e-4cc2-baac-3cd19fcc579c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:16.912976 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b980f45a-703e-4cc2-baac-3cd19fcc579c None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:16.935159 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application [None req-b980f45a-703e-4cc2-baac-3cd19fcc579c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 11:58:16.935159 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 11:58:16.935159 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 11:58:16.935159 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 11:58:16.935159 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 11:58:16.935159 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 11:58:16.935159 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 11:58:16.935159 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 11:58:16.935159 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 11:58:16.935159 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 11:58:16.935159 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 11:58:16.935159 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 11:58:16.935159 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 11:58:16.935159 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 11:58:16.935159 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 11:58:16.937150 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 11:58:16.937150 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 11:58:16.937150 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 11:58:16.937150 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 11:58:16.937150 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 11:58:16.937150 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 11:58:16.937150 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.get_or_create( May 19 11:58:16.937150 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 11:58:16.937150 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application with Lock( May 19 11:58:16.937150 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 11:58:16.937150 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._enter() May 19 11:58:16.937150 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 11:58:16.937150 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 11:58:16.937150 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 11:58:16.937150 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.creator() May 19 11:58:16.937150 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 11:58:16.937150 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application created_value = creator( May 19 11:58:16.938276 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 11:58:16.938276 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 11:58:16.938276 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 11:58:16.938276 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application May 19 11:58:16.940253 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 315/624] 10.222.0.23 () {56 vars in 1108 bytes} [Thu May 19 11:58:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 11:58:16.940846 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e2efa7-9e9d-4d46-b0c7-4b35198c3771 None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:16.941542 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b980f45a-703e-4cc2-baac-3cd19fcc579c None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:16.984939 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dcd3878-0e2b-4920-9d02-70f6c1ed8df2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:16.985331 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcd3878-0e2b-4920-9d02-70f6c1ed8df2 None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:16.985621 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcd3878-0e2b-4920-9d02-70f6c1ed8df2 None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:16.985900 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcd3878-0e2b-4920-9d02-70f6c1ed8df2 None admin] PATH_INFO: `/v3/roles` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:16.986643 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dcd3878-0e2b-4920-9d02-70f6c1ed8df2 None admin] RBAC: Adding query filter params (name=service) {{(pid=80539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 11:58:16.986949 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dcd3878-0e2b-4920-9d02-70f6c1ed8df2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:16.988346 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dcd3878-0e2b-4920-9d02-70f6c1ed8df2 None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:17.002510 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 310/625] 10.222.0.23 () {56 vars in 1103 bytes} [Thu May 19 11:58:16 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:58:17.002510 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dcd3878-0e2b-4920-9d02-70f6c1ed8df2 None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:17.005719 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:58:17.059612 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b980f45a-703e-4cc2-baac-3cd19fcc579c None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:17.096974 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b23b01-92b3-4e89-87f1-56996e15d241 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:17.098255 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b23b01-92b3-4e89-87f1-56996e15d241 None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:17.098719 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b23b01-92b3-4e89-87f1-56996e15d241 None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:17.099144 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b23b01-92b3-4e89-87f1-56996e15d241 None admin] PATH_INFO: `/v3/domains` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:17.100045 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b23b01-92b3-4e89-87f1-56996e15d241 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 11:58:17.100496 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b23b01-92b3-4e89-87f1-56996e15d241 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:17.102035 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b23b01-92b3-4e89-87f1-56996e15d241 None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:17.116242 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 316/626] 10.222.0.23 () {56 vars in 1109 bytes} [Thu May 19 11:58:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:58:17.117464 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b23b01-92b3-4e89-87f1-56996e15d241 None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:17.120942 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dcd3878-0e2b-4920-9d02-70f6c1ed8df2 None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:17.123993 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:58:17.165908 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907916fe-9678-416c-a291-043acf8e2b88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:17.166588 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907916fe-9678-416c-a291-043acf8e2b88 None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:17.166875 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907916fe-9678-416c-a291-043acf8e2b88 None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:17.167154 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907916fe-9678-416c-a291-043acf8e2b88 None admin] PATH_INFO: `/v3/domains` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:17.167966 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907916fe-9678-416c-a291-043acf8e2b88 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 11:58:17.168285 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907916fe-9678-416c-a291-043acf8e2b88 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:17.169974 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907916fe-9678-416c-a291-043acf8e2b88 None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:17.186819 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 311/627] 10.222.0.23 () {56 vars in 1109 bytes} [Thu May 19 11:58:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:58:17.187693 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907916fe-9678-416c-a291-043acf8e2b88 None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:17.235107 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b23b01-92b3-4e89-87f1-56996e15d241 None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:17.246654 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:58:17.273180 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418b0ba7-825a-43d7-86ad-ac6c94737b8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:17.274468 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418b0ba7-825a-43d7-86ad-ac6c94737b8e None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:17.275086 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418b0ba7-825a-43d7-86ad-ac6c94737b8e None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:17.275716 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418b0ba7-825a-43d7-86ad-ac6c94737b8e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:17.287579 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418b0ba7-825a-43d7-86ad-ac6c94737b8e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:17.289424 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418b0ba7-825a-43d7-86ad-ac6c94737b8e None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:17.308320 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application [None req-418b0ba7-825a-43d7-86ad-ac6c94737b8e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 11:58:17.308320 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 11:58:17.308320 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 11:58:17.308320 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 11:58:17.308320 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 11:58:17.308320 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 11:58:17.308320 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 11:58:17.308320 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 11:58:17.308320 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 11:58:17.308320 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 11:58:17.308320 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 11:58:17.308320 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 11:58:17.308320 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 11:58:17.308320 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 11:58:17.308320 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 11:58:17.309253 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 11:58:17.309253 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 11:58:17.309253 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 11:58:17.309253 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 11:58:17.309253 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 11:58:17.309253 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 11:58:17.309253 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.get_or_create( May 19 11:58:17.309253 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 11:58:17.309253 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application with Lock( May 19 11:58:17.309253 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 11:58:17.309253 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._enter() May 19 11:58:17.309253 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 11:58:17.309253 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 11:58:17.309253 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 11:58:17.309253 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.creator() May 19 11:58:17.309253 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 11:58:17.309253 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application created_value = creator( May 19 11:58:17.310216 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 11:58:17.310216 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 11:58:17.310216 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 11:58:17.310216 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application May 19 11:58:17.312508 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907916fe-9678-416c-a291-043acf8e2b88 None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:17.317839 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 317/628] 10.222.0.23 () {56 vars in 1108 bytes} [Thu May 19 11:58:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 11:58:17.319008 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418b0ba7-825a-43d7-86ad-ac6c94737b8e None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:17.351342 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6816a561-1efc-414c-ac1f-30ea55cefc28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:17.352224 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6816a561-1efc-414c-ac1f-30ea55cefc28 None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:17.352459 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6816a561-1efc-414c-ac1f-30ea55cefc28 None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:17.352739 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6816a561-1efc-414c-ac1f-30ea55cefc28 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:17.359747 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6816a561-1efc-414c-ac1f-30ea55cefc28 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:17.361232 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6816a561-1efc-414c-ac1f-30ea55cefc28 None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:17.375586 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application [None req-6816a561-1efc-414c-ac1f-30ea55cefc28 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 11:58:17.375586 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 11:58:17.375586 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 11:58:17.375586 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 11:58:17.375586 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 11:58:17.375586 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 11:58:17.375586 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 11:58:17.375586 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 11:58:17.375586 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 11:58:17.375586 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 11:58:17.375586 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 11:58:17.375586 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 11:58:17.375586 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 11:58:17.375586 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 11:58:17.375586 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 11:58:17.376544 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 11:58:17.376544 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 11:58:17.376544 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 11:58:17.376544 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 11:58:17.376544 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 11:58:17.376544 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 11:58:17.376544 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.get_or_create( May 19 11:58:17.376544 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 11:58:17.376544 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application with Lock( May 19 11:58:17.376544 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 11:58:17.376544 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._enter() May 19 11:58:17.376544 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 11:58:17.376544 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 11:58:17.376544 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 11:58:17.376544 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.creator() May 19 11:58:17.376544 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 11:58:17.376544 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application created_value = creator( May 19 11:58:17.377398 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 11:58:17.377398 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 11:58:17.377398 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 11:58:17.377398 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 11:58:17.377398 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 11:58:17.377398 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 11:58:17.377398 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 11:58:17.377398 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application May 19 11:58:17.380064 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 312/629] 10.222.0.23 () {56 vars in 1111 bytes} [Thu May 19 11:58:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 11:58:17.381151 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6816a561-1efc-414c-ac1f-30ea55cefc28 None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:17.409249 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:58:17.431878 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418b0ba7-825a-43d7-86ad-ac6c94737b8e None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:17.479363 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe680788-9a55-4d30-8506-f1aad5157ada None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:17.481588 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe680788-9a55-4d30-8506-f1aad5157ada None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:17.482060 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe680788-9a55-4d30-8506-f1aad5157ada None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:17.483029 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe680788-9a55-4d30-8506-f1aad5157ada None admin] PATH_INFO: `/v3/users/nova` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:17.494522 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe680788-9a55-4d30-8506-f1aad5157ada None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:17.497131 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe680788-9a55-4d30-8506-f1aad5157ada None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:17.499754 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6816a561-1efc-414c-ac1f-30ea55cefc28 None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:17.517693 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application [None req-fe680788-9a55-4d30-8506-f1aad5157ada None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 19 11:58:17.517693 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 11:58:17.517693 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 11:58:17.517693 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 11:58:17.517693 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 11:58:17.517693 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 11:58:17.517693 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 11:58:17.517693 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 11:58:17.517693 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 11:58:17.517693 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 11:58:17.517693 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 11:58:17.517693 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 11:58:17.517693 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 11:58:17.517693 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 11:58:17.517693 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 11:58:17.517693 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 11:58:17.518659 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 11:58:17.518659 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 11:58:17.518659 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 11:58:17.518659 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 11:58:17.518659 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 11:58:17.518659 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 11:58:17.518659 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 11:58:17.518659 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 11:58:17.518659 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 11:58:17.518659 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.get_or_create( May 19 11:58:17.518659 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 11:58:17.518659 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application with Lock( May 19 11:58:17.518659 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 11:58:17.518659 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._enter() May 19 11:58:17.518659 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 11:58:17.518659 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 11:58:17.518659 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 11:58:17.518659 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.creator() May 19 11:58:17.519577 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 11:58:17.519577 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application created_value = creator( May 19 11:58:17.519577 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 11:58:17.519577 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 11:58:17.519577 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 11:58:17.519577 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 11:58:17.519577 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 11:58:17.519577 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 11:58:17.519577 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 19 11:58:17.519577 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application May 19 11:58:17.521733 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 318/630] 10.222.0.23 () {56 vars in 1093 bytes} [Thu May 19 11:58:17 2022] GET /identity/v3/users/nova => generated 82 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 11:58:17.522828 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd923e5c-1e47-4da4-973f-07b14b76e30a None None] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:17.523116 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd923e5c-1e47-4da4-973f-07b14b76e30a None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:17.523393 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd923e5c-1e47-4da4-973f-07b14b76e30a None None] PATH_INFO: `/` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:17.524969 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 319/631] 10.222.0.23 () {54 vars in 926 bytes} [Thu May 19 11:58:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 11:58:17.525733 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd923e5c-1e47-4da4-973f-07b14b76e30a None None] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:17.546713 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b097a629-9adc-4ba6-98c1-03960b3aa73d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:17.547798 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b097a629-9adc-4ba6-98c1-03960b3aa73d None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:17.548084 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b097a629-9adc-4ba6-98c1-03960b3aa73d None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:17.548360 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b097a629-9adc-4ba6-98c1-03960b3aa73d None admin] PATH_INFO: `/v3/domains` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:17.549086 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b097a629-9adc-4ba6-98c1-03960b3aa73d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 11:58:17.549390 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b097a629-9adc-4ba6-98c1-03960b3aa73d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:17.550151 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:58:17.551309 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b097a629-9adc-4ba6-98c1-03960b3aa73d None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:17.571937 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 313/632] 10.222.0.23 () {56 vars in 1109 bytes} [Thu May 19 11:58:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:58:17.573202 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b097a629-9adc-4ba6-98c1-03960b3aa73d None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:17.679337 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd923e5c-1e47-4da4-973f-07b14b76e30a None None] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:17.700134 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:58:17.718309 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b097a629-9adc-4ba6-98c1-03960b3aa73d None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:17.721085 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3883b32c-0abc-4dde-8ff3-3be76b079076 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:17.721757 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3883b32c-0abc-4dde-8ff3-3be76b079076 None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:17.722047 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3883b32c-0abc-4dde-8ff3-3be76b079076 None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:17.722423 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3883b32c-0abc-4dde-8ff3-3be76b079076 None admin] PATH_INFO: `/v3/projects` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:17.723127 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3883b32c-0abc-4dde-8ff3-3be76b079076 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 11:58:17.723441 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3883b32c-0abc-4dde-8ff3-3be76b079076 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:17.724941 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3883b32c-0abc-4dde-8ff3-3be76b079076 None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:17.742016 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 320/633] 10.222.0.23 () {56 vars in 1148 bytes} [Thu May 19 11:58:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:58:17.743661 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843312ca-1ef7-4768-9017-d3e0b6db2aee None None] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:17.743977 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843312ca-1ef7-4768-9017-d3e0b6db2aee None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:17.744263 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843312ca-1ef7-4768-9017-d3e0b6db2aee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:17.760816 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ed8b4a-6997-4804-bfed-ec288c218e56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:17.761500 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ed8b4a-6997-4804-bfed-ec288c218e56 None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:17.762019 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ed8b4a-6997-4804-bfed-ec288c218e56 None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:17.762379 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ed8b4a-6997-4804-bfed-ec288c218e56 None admin] PATH_INFO: `/v3/users` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:17.763568 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ed8b4a-6997-4804-bfed-ec288c218e56 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 11:58:17.764502 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ed8b4a-6997-4804-bfed-ec288c218e56 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:17.765944 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ed8b4a-6997-4804-bfed-ec288c218e56 None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:17.774300 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.auth.core [None req-843312ca-1ef7-4768-9017-d3e0b6db2aee None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 11:58:17.778044 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-843312ca-1ef7-4768-9017-d3e0b6db2aee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 11:58:17.789628 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 314/634] 10.222.0.23 () {56 vars in 1133 bytes} [Thu May 19 11:58:17 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:58:17.789628 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ed8b4a-6997-4804-bfed-ec288c218e56 None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:17.895819 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 321/635] 10.222.0.23 () {58 vars in 1023 bytes} [Thu May 19 11:58:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 11:58:17.895819 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843312ca-1ef7-4768-9017-d3e0b6db2aee None None] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:17.906453 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:58:17.931659 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ed8b4a-6997-4804-bfed-ec288c218e56 None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:17.969901 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac64662-84b4-4330-ac97-ff893d7fa69b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:17.970748 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac64662-84b4-4330-ac97-ff893d7fa69b None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:17.971038 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac64662-84b4-4330-ac97-ff893d7fa69b None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:17.971319 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac64662-84b4-4330-ac97-ff893d7fa69b None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:17.979722 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac64662-84b4-4330-ac97-ff893d7fa69b None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:17.981420 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac64662-84b4-4330-ac97-ff893d7fa69b None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:17.997456 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application [None req-3ac64662-84b4-4330-ac97-ff893d7fa69b None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 19 11:58:17.997456 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 11:58:17.997456 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 11:58:17.997456 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 11:58:17.997456 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 11:58:17.997456 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 11:58:17.997456 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 11:58:17.997456 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 11:58:17.997456 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 11:58:17.997456 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 11:58:17.997456 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 11:58:17.997456 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 11:58:17.997456 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 11:58:17.997456 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 11:58:17.997456 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 11:58:17.997456 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 11:58:17.998958 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 11:58:17.998958 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 11:58:17.998958 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 11:58:17.998958 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 11:58:17.998958 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 11:58:17.998958 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 11:58:17.998958 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 11:58:17.998958 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 11:58:17.998958 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 11:58:17.998958 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.get_or_create( May 19 11:58:17.998958 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 11:58:17.998958 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application with Lock( May 19 11:58:17.998958 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 11:58:17.998958 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._enter() May 19 11:58:17.998958 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 11:58:17.998958 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 11:58:17.998958 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 11:58:17.998958 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.creator() May 19 11:58:17.999910 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 11:58:17.999910 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application created_value = creator( May 19 11:58:17.999910 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 11:58:17.999910 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 11:58:17.999910 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 11:58:17.999910 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 11:58:17.999910 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 11:58:17.999910 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 11:58:17.999910 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 19 11:58:17.999910 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application May 19 11:58:18.004149 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 315/636] 10.222.0.23 () {56 vars in 1099 bytes} [Thu May 19 11:58:17 2022] GET /identity/v3/users/cinder => generated 84 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 11:58:18.004893 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac64662-84b4-4330-ac97-ff893d7fa69b None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:18.007613 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:58:18.015295 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843312ca-1ef7-4768-9017-d3e0b6db2aee None None] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:18.053466 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3119d1-f461-4387-b917-40ea18148c5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:18.054352 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3119d1-f461-4387-b917-40ea18148c5d None admin] REQUEST_METHOD: `PUT` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:18.054638 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3119d1-f461-4387-b917-40ea18148c5d None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:18.054917 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3119d1-f461-4387-b917-40ea18148c5d None admin] PATH_INFO: `/v3/projects/d70d25bf69314820b8aaecb65576978a/users/15cbfc73ae1243d2a8b1e70f9cd99876/roles/7c59061ba7064a0db6032433fb0eec6b` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:18.071181 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3119d1-f461-4387-b917-40ea18148c5d None admin] RBAC: Authorizing `identity:create_grant(project_id=d70d25bf69314820b8aaecb65576978a, user_id=15cbfc73ae1243d2a8b1e70f9cd99876, role_id=7c59061ba7064a0db6032433fb0eec6b)` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:18.072915 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3119d1-f461-4387-b917-40ea18148c5d None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:18.113801 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac64662-84b4-4330-ac97-ff893d7fa69b None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:18.142545 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 322/637] 10.222.0.23 () {58 vars in 1439 bytes} [Thu May 19 11:58:17 2022] PUT /identity/v3/projects/d70d25bf69314820b8aaecb65576978a/users/15cbfc73ae1243d2a8b1e70f9cd99876/roles/7c59061ba7064a0db6032433fb0eec6b => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 11:58:18.143414 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3119d1-f461-4387-b917-40ea18148c5d None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:18.165802 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8e92f6-381f-43b4-a534-f2574699534a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:18.167377 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8e92f6-381f-43b4-a534-f2574699534a None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:18.168627 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8e92f6-381f-43b4-a534-f2574699534a None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:18.169214 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8e92f6-381f-43b4-a534-f2574699534a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:18.224659 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:58:18.253444 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea8e92f6-381f-43b4-a534-f2574699534a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:18.254880 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea8e92f6-381f-43b4-a534-f2574699534a None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:18.257266 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:58:18.274694 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3119d1-f461-4387-b917-40ea18148c5d None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:18.320736 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb66d4fc-bf4e-4d31-9ef5-8b0226f43bcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:18.322112 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb66d4fc-bf4e-4d31-9ef5-8b0226f43bcc None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:18.322700 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb66d4fc-bf4e-4d31-9ef5-8b0226f43bcc None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:18.323193 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb66d4fc-bf4e-4d31-9ef5-8b0226f43bcc None admin] PATH_INFO: `/v3/roles/service` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:18.339032 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb66d4fc-bf4e-4d31-9ef5-8b0226f43bcc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 11:58:18.348566 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application [None req-fb66d4fc-bf4e-4d31-9ef5-8b0226f43bcc None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 11:58:18.348566 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 11:58:18.348566 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 11:58:18.348566 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 11:58:18.348566 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 11:58:18.348566 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 11:58:18.348566 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 11:58:18.348566 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 11:58:18.348566 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 11:58:18.348566 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 11:58:18.348566 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 11:58:18.348566 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 11:58:18.348566 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 11:58:18.348566 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 11:58:18.348566 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 11:58:18.348566 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 11:58:18.349564 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 11:58:18.349564 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 11:58:18.349564 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 11:58:18.349564 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 11:58:18.349564 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 11:58:18.349564 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 11:58:18.349564 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 11:58:18.349564 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 11:58:18.349564 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 11:58:18.349564 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.get_or_create( May 19 11:58:18.349564 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 11:58:18.349564 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application with Lock( May 19 11:58:18.349564 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 11:58:18.349564 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._enter() May 19 11:58:18.349564 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 11:58:18.349564 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 11:58:18.350395 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 11:58:18.350395 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.creator() May 19 11:58:18.350395 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 11:58:18.350395 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application created_value = creator( May 19 11:58:18.350395 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 11:58:18.350395 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 11:58:18.350395 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 11:58:18.350395 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 11:58:18.350395 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 11:58:18.350395 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 11:58:18.350395 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 11:58:18.350395 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application May 19 11:58:18.354940 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 323/638] 10.222.0.23 () {56 vars in 1102 bytes} [Thu May 19 11:58:18 2022] GET /identity/v3/roles/service => generated 85 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 11:58:18.355955 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb66d4fc-bf4e-4d31-9ef5-8b0226f43bcc None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:18.390620 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 316/639] 10.222.0.23 () {58 vars in 1282 bytes} [Thu May 19 11:58:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 11:58:18.391904 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd64fca-18df-42aa-ad7b-8db0ccead79e None None] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:18.392198 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd64fca-18df-42aa-ad7b-8db0ccead79e None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:18.392481 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd64fca-18df-42aa-ad7b-8db0ccead79e None None] PATH_INFO: `/` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:18.393951 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 317/640] 10.222.0.23 () {54 vars in 926 bytes} [Thu May 19 11:58:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 11:58:18.394528 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd64fca-18df-42aa-ad7b-8db0ccead79e None None] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:18.484294 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb66d4fc-bf4e-4d31-9ef5-8b0226f43bcc None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:18.512391 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:58:18.515427 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd64fca-18df-42aa-ad7b-8db0ccead79e None None] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:18.517517 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:58:18.527632 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ecbcfdd-6ec9-48b1-ab6a-7b6c9e90ea20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:18.528857 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecbcfdd-6ec9-48b1-ab6a-7b6c9e90ea20 None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:18.528857 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecbcfdd-6ec9-48b1-ab6a-7b6c9e90ea20 None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:18.529119 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecbcfdd-6ec9-48b1-ab6a-7b6c9e90ea20 None admin] PATH_INFO: `/v3/users` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:18.530022 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ecbcfdd-6ec9-48b1-ab6a-7b6c9e90ea20 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 11:58:18.531008 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ecbcfdd-6ec9-48b1-ab6a-7b6c9e90ea20 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:18.531856 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ecbcfdd-6ec9-48b1-ab6a-7b6c9e90ea20 None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:18.556878 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 324/641] 10.222.0.23 () {56 vars in 1137 bytes} [Thu May 19 11:58:18 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:58:18.557139 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9b0539-df40-4a11-b85a-f9107c1bc8a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:18.557978 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9b0539-df40-4a11-b85a-f9107c1bc8a1 None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:18.559485 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ecbcfdd-6ec9-48b1-ab6a-7b6c9e90ea20 None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:18.559599 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9b0539-df40-4a11-b85a-f9107c1bc8a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:18.559599 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9b0539-df40-4a11-b85a-f9107c1bc8a1 None admin] PATH_INFO: `/v3/roles` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:18.559599 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9b0539-df40-4a11-b85a-f9107c1bc8a1 None admin] RBAC: Adding query filter params (name=service) {{(pid=80539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 11:58:18.559599 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9b0539-df40-4a11-b85a-f9107c1bc8a1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:18.561696 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9b0539-df40-4a11-b85a-f9107c1bc8a1 None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:18.579660 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 318/642] 10.222.0.23 () {56 vars in 1103 bytes} [Thu May 19 11:58:18 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:58:18.581134 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb5108f-415e-4b94-8a6e-49c81a9036d5 None None] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:18.581428 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb5108f-415e-4b94-8a6e-49c81a9036d5 None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:18.581709 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb5108f-415e-4b94-8a6e-49c81a9036d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:18.603584 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.auth.core [None req-8eb5108f-415e-4b94-8a6e-49c81a9036d5 None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 11:58:18.604896 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-8eb5108f-415e-4b94-8a6e-49c81a9036d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 11:58:18.681206 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ecbcfdd-6ec9-48b1-ab6a-7b6c9e90ea20 None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:18.683825 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 319/643] 10.222.0.23 () {58 vars in 1023 bytes} [Thu May 19 11:58:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 11:58:18.683825 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb5108f-415e-4b94-8a6e-49c81a9036d5 None None] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:18.726459 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d223b2-0d13-4b1b-923a-3d955188a99e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:18.727266 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d223b2-0d13-4b1b-923a-3d955188a99e None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:18.727553 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d223b2-0d13-4b1b-923a-3d955188a99e None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:18.727856 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d223b2-0d13-4b1b-923a-3d955188a99e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:18.742989 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d223b2-0d13-4b1b-923a-3d955188a99e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:18.744840 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d223b2-0d13-4b1b-923a-3d955188a99e None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:18.763533 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application [None req-b2d223b2-0d13-4b1b-923a-3d955188a99e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 11:58:18.763533 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 11:58:18.763533 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 11:58:18.763533 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 11:58:18.763533 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 11:58:18.763533 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 11:58:18.763533 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 11:58:18.763533 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 11:58:18.763533 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 11:58:18.763533 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 11:58:18.763533 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 11:58:18.763533 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 11:58:18.763533 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 11:58:18.763533 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 11:58:18.763533 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 11:58:18.764799 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 11:58:18.764799 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 11:58:18.764799 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 11:58:18.764799 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 11:58:18.764799 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 11:58:18.764799 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 11:58:18.764799 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.get_or_create( May 19 11:58:18.764799 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 11:58:18.764799 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application with Lock( May 19 11:58:18.764799 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 11:58:18.764799 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._enter() May 19 11:58:18.764799 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 11:58:18.764799 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 11:58:18.764799 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 11:58:18.764799 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.creator() May 19 11:58:18.764799 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 11:58:18.764799 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application created_value = creator( May 19 11:58:18.770700 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 11:58:18.770700 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 11:58:18.770700 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 11:58:18.770700 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application May 19 11:58:18.770700 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 325/644] 10.222.0.23 () {56 vars in 1108 bytes} [Thu May 19 11:58:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 11:58:18.770700 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d223b2-0d13-4b1b-923a-3d955188a99e None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:18.801765 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:58:18.804047 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:58:18.811463 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb5108f-415e-4b94-8a6e-49c81a9036d5 None None] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:18.851386 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb74c96a-7fb9-4dd1-8b31-49a0971cd11c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:18.852071 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb74c96a-7fb9-4dd1-8b31-49a0971cd11c None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:18.852356 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb74c96a-7fb9-4dd1-8b31-49a0971cd11c None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:18.852662 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb74c96a-7fb9-4dd1-8b31-49a0971cd11c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:18.878679 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d223b2-0d13-4b1b-923a-3d955188a99e None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:18.916796 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ad5205-1c09-4d30-b304-9e02ec9f6bd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:18.917899 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ad5205-1c09-4d30-b304-9e02ec9f6bd9 None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:18.918376 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ad5205-1c09-4d30-b304-9e02ec9f6bd9 None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:18.918863 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ad5205-1c09-4d30-b304-9e02ec9f6bd9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:18.923254 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb74c96a-7fb9-4dd1-8b31-49a0971cd11c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:18.924749 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb74c96a-7fb9-4dd1-8b31-49a0971cd11c None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:18.932233 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ad5205-1c09-4d30-b304-9e02ec9f6bd9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:18.934181 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ad5205-1c09-4d30-b304-9e02ec9f6bd9 None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:18.954643 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application [None req-40ad5205-1c09-4d30-b304-9e02ec9f6bd9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 11:58:18.954643 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 11:58:18.954643 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 11:58:18.954643 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 11:58:18.954643 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 11:58:18.954643 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 11:58:18.954643 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 11:58:18.954643 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 11:58:18.954643 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 11:58:18.954643 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 11:58:18.954643 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 11:58:18.954643 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 11:58:18.954643 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 11:58:18.954643 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 11:58:18.954643 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 11:58:18.955527 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 11:58:18.955527 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 11:58:18.955527 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 11:58:18.955527 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 11:58:18.955527 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 11:58:18.955527 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 11:58:18.955527 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.get_or_create( May 19 11:58:18.955527 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 11:58:18.955527 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application with Lock( May 19 11:58:18.955527 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 11:58:18.955527 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._enter() May 19 11:58:18.955527 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 11:58:18.955527 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 11:58:18.955527 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 11:58:18.955527 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.creator() May 19 11:58:18.955527 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 11:58:18.955527 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application created_value = creator( May 19 11:58:18.956615 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 11:58:18.956615 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 11:58:18.956615 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 11:58:18.956615 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application May 19 11:58:18.956615 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 326/645] 10.222.0.23 () {56 vars in 1108 bytes} [Thu May 19 11:58:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 11:58:18.956910 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ad5205-1c09-4d30-b304-9e02ec9f6bd9 None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:19.033419 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:58:19.044823 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 320/646] 10.222.0.23 () {58 vars in 1282 bytes} [Thu May 19 11:58:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 11:58:19.044823 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb74c96a-7fb9-4dd1-8b31-49a0971cd11c None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:19.050299 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:58:19.069163 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ad5205-1c09-4d30-b304-9e02ec9f6bd9 None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:19.108243 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf44056-aeb2-48a1-92a5-10b198539345 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:19.109049 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf44056-aeb2-48a1-92a5-10b198539345 None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:19.109337 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf44056-aeb2-48a1-92a5-10b198539345 None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:19.109621 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf44056-aeb2-48a1-92a5-10b198539345 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:19.120860 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf44056-aeb2-48a1-92a5-10b198539345 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 11:58:19.130007 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application [None req-faf44056-aeb2-48a1-92a5-10b198539345 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 11:58:19.130007 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 11:58:19.130007 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 11:58:19.130007 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 11:58:19.130007 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 11:58:19.130007 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 11:58:19.130007 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 11:58:19.130007 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 11:58:19.130007 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 11:58:19.130007 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 11:58:19.130007 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 11:58:19.130007 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 11:58:19.130007 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 11:58:19.130007 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 11:58:19.130007 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 11:58:19.130007 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 11:58:19.131014 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 11:58:19.131014 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 11:58:19.131014 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 11:58:19.131014 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 11:58:19.131014 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 11:58:19.131014 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 11:58:19.131014 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 11:58:19.131014 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 11:58:19.131014 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 11:58:19.131014 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.get_or_create( May 19 11:58:19.131014 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 11:58:19.131014 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application with Lock( May 19 11:58:19.131014 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 11:58:19.131014 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._enter() May 19 11:58:19.131014 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 11:58:19.131014 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 11:58:19.131876 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 11:58:19.131876 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.creator() May 19 11:58:19.131876 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 11:58:19.131876 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application created_value = creator( May 19 11:58:19.131876 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 11:58:19.131876 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 11:58:19.131876 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 11:58:19.131876 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 11:58:19.131876 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 11:58:19.131876 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 11:58:19.131876 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 11:58:19.131876 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application May 19 11:58:19.131876 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 327/647] 10.222.0.23 () {56 vars in 1102 bytes} [Thu May 19 11:58:18 2022] GET /identity/v3/roles/service => generated 85 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 11:58:19.132624 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf44056-aeb2-48a1-92a5-10b198539345 None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:19.160411 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb74c96a-7fb9-4dd1-8b31-49a0971cd11c None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:19.196584 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620efdd6-baae-4147-932b-05fdbec5c7d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:19.197243 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620efdd6-baae-4147-932b-05fdbec5c7d9 None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:19.197539 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620efdd6-baae-4147-932b-05fdbec5c7d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:19.197789 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620efdd6-baae-4147-932b-05fdbec5c7d9 None admin] PATH_INFO: `/v3/domains` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:19.198587 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620efdd6-baae-4147-932b-05fdbec5c7d9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 11:58:19.198810 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620efdd6-baae-4147-932b-05fdbec5c7d9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:19.200216 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620efdd6-baae-4147-932b-05fdbec5c7d9 None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:19.217668 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 321/648] 10.222.0.23 () {56 vars in 1109 bytes} [Thu May 19 11:58:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:58:19.217668 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620efdd6-baae-4147-932b-05fdbec5c7d9 None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:19.246197 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf44056-aeb2-48a1-92a5-10b198539345 None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:19.276149 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:58:19.291694 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c02fb5-a890-4030-b93c-cc34c3ac0030 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:19.291694 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c02fb5-a890-4030-b93c-cc34c3ac0030 None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:19.291694 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c02fb5-a890-4030-b93c-cc34c3ac0030 None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:19.291694 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c02fb5-a890-4030-b93c-cc34c3ac0030 None admin] PATH_INFO: `/v3/domains` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:19.292226 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c02fb5-a890-4030-b93c-cc34c3ac0030 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 11:58:19.292226 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c02fb5-a890-4030-b93c-cc34c3ac0030 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:19.292226 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c02fb5-a890-4030-b93c-cc34c3ac0030 None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:19.301273 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 328/649] 10.222.0.23 () {56 vars in 1109 bytes} [Thu May 19 11:58:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:58:19.304766 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c02fb5-a890-4030-b93c-cc34c3ac0030 None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:19.307968 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:58:19.324821 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620efdd6-baae-4147-932b-05fdbec5c7d9 None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:19.375310 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d1e50e-0280-42f6-b282-6b17c80b6f02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:19.376480 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d1e50e-0280-42f6-b282-6b17c80b6f02 None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:19.376649 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d1e50e-0280-42f6-b282-6b17c80b6f02 None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:19.376843 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d1e50e-0280-42f6-b282-6b17c80b6f02 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:19.389572 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d1e50e-0280-42f6-b282-6b17c80b6f02 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:19.392400 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d1e50e-0280-42f6-b282-6b17c80b6f02 None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:19.422825 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application [None req-20d1e50e-0280-42f6-b282-6b17c80b6f02 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 11:58:19.422825 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 11:58:19.422825 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 11:58:19.422825 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 11:58:19.422825 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 11:58:19.422825 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 11:58:19.422825 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 11:58:19.422825 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 11:58:19.422825 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 11:58:19.422825 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 11:58:19.422825 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 11:58:19.422825 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 11:58:19.422825 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 11:58:19.422825 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 11:58:19.422825 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 11:58:19.423832 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 11:58:19.423832 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 11:58:19.423832 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 11:58:19.423832 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 11:58:19.423832 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 11:58:19.423832 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 11:58:19.423832 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.get_or_create( May 19 11:58:19.423832 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 11:58:19.423832 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application with Lock( May 19 11:58:19.423832 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 11:58:19.423832 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._enter() May 19 11:58:19.423832 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 11:58:19.423832 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 11:58:19.423832 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 11:58:19.423832 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.creator() May 19 11:58:19.423832 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 11:58:19.423832 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application created_value = creator( May 19 11:58:19.424874 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 11:58:19.424874 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 11:58:19.424874 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 11:58:19.424874 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application May 19 11:58:19.424874 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 322/650] 10.222.0.23 () {56 vars in 1108 bytes} [Thu May 19 11:58:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 11:58:19.425909 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d1e50e-0280-42f6-b282-6b17c80b6f02 None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:19.449819 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c02fb5-a890-4030-b93c-cc34c3ac0030 None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:19.499036 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69681979-0625-42e7-933c-da59db1bb5f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:19.499773 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69681979-0625-42e7-933c-da59db1bb5f6 None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:19.500164 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69681979-0625-42e7-933c-da59db1bb5f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:19.500510 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69681979-0625-42e7-933c-da59db1bb5f6 None admin] PATH_INFO: `/v3/roles` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:19.501283 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69681979-0625-42e7-933c-da59db1bb5f6 None admin] RBAC: Adding query filter params (name=service) {{(pid=80540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 11:58:19.501595 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69681979-0625-42e7-933c-da59db1bb5f6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:19.503004 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69681979-0625-42e7-933c-da59db1bb5f6 None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:19.517512 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 329/651] 10.222.0.23 () {56 vars in 1103 bytes} [Thu May 19 11:58:19 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:58:19.517512 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69681979-0625-42e7-933c-da59db1bb5f6 None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:19.563930 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d1e50e-0280-42f6-b282-6b17c80b6f02 None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:19.572686 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:58:19.589326 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:58:19.606697 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20317b62-f393-44ea-aa1e-edcd23d81ab9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:19.607750 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20317b62-f393-44ea-aa1e-edcd23d81ab9 None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:19.608204 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20317b62-f393-44ea-aa1e-edcd23d81ab9 None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:19.608735 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20317b62-f393-44ea-aa1e-edcd23d81ab9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:19.618362 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20317b62-f393-44ea-aa1e-edcd23d81ab9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:19.619891 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20317b62-f393-44ea-aa1e-edcd23d81ab9 None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:19.636316 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application [None req-20317b62-f393-44ea-aa1e-edcd23d81ab9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 11:58:19.636316 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 11:58:19.636316 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 11:58:19.636316 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 11:58:19.636316 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 11:58:19.636316 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 11:58:19.636316 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 11:58:19.636316 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 11:58:19.636316 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 11:58:19.636316 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 11:58:19.636316 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 11:58:19.636316 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 11:58:19.636316 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 11:58:19.636316 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 11:58:19.636316 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 11:58:19.637426 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 11:58:19.637426 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 11:58:19.637426 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 11:58:19.637426 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 11:58:19.637426 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 11:58:19.637426 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 11:58:19.637426 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.get_or_create( May 19 11:58:19.637426 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 11:58:19.637426 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application with Lock( May 19 11:58:19.637426 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 11:58:19.637426 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._enter() May 19 11:58:19.637426 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 11:58:19.637426 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 11:58:19.637426 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 11:58:19.637426 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.creator() May 19 11:58:19.637426 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 11:58:19.637426 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application created_value = creator( May 19 11:58:19.638397 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 11:58:19.638397 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 11:58:19.638397 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 11:58:19.638397 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 11:58:19.638397 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 11:58:19.638397 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 11:58:19.638397 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 11:58:19.638397 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application May 19 11:58:19.638397 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 323/652] 10.222.0.23 () {56 vars in 1111 bytes} [Thu May 19 11:58:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 11:58:19.639507 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20317b62-f393-44ea-aa1e-edcd23d81ab9 None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:19.641404 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69681979-0625-42e7-933c-da59db1bb5f6 None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:19.681940 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a4643d-415b-4e98-a67a-0402c98c7790 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:19.682796 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a4643d-415b-4e98-a67a-0402c98c7790 None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:19.683179 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a4643d-415b-4e98-a67a-0402c98c7790 None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:19.683365 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a4643d-415b-4e98-a67a-0402c98c7790 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:19.691468 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a4643d-415b-4e98-a67a-0402c98c7790 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:19.693134 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a4643d-415b-4e98-a67a-0402c98c7790 None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:19.709673 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application [None req-84a4643d-415b-4e98-a67a-0402c98c7790 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. May 19 11:58:19.709673 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 11:58:19.709673 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 11:58:19.709673 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 11:58:19.709673 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 11:58:19.709673 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 11:58:19.709673 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 11:58:19.709673 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 11:58:19.709673 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 11:58:19.709673 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 11:58:19.709673 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 11:58:19.709673 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 11:58:19.709673 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 11:58:19.709673 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 11:58:19.709673 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 11:58:19.709673 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 11:58:19.711931 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 11:58:19.711931 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 11:58:19.711931 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 11:58:19.711931 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 11:58:19.711931 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 11:58:19.711931 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 11:58:19.711931 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 11:58:19.711931 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 11:58:19.711931 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 11:58:19.711931 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.get_or_create( May 19 11:58:19.711931 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 11:58:19.711931 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application with Lock( May 19 11:58:19.711931 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 11:58:19.711931 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._enter() May 19 11:58:19.711931 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 11:58:19.711931 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 11:58:19.711931 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 11:58:19.711931 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.creator() May 19 11:58:19.712855 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 11:58:19.712855 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application created_value = creator( May 19 11:58:19.712855 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 11:58:19.712855 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 11:58:19.712855 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 11:58:19.712855 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 11:58:19.712855 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 11:58:19.712855 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 11:58:19.712855 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. May 19 11:58:19.712855 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application May 19 11:58:19.714281 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 330/653] 10.222.0.23 () {56 vars in 1099 bytes} [Thu May 19 11:58:19 2022] GET /identity/v3/users/glance => generated 84 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 11:58:19.715885 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a4643d-415b-4e98-a67a-0402c98c7790 None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:19.762287 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20317b62-f393-44ea-aa1e-edcd23d81ab9 None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:19.801095 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24696cc0-77cd-4edb-a968-49ee9bd2cd4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:19.801768 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24696cc0-77cd-4edb-a968-49ee9bd2cd4d None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:19.802288 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24696cc0-77cd-4edb-a968-49ee9bd2cd4d None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:19.802581 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24696cc0-77cd-4edb-a968-49ee9bd2cd4d None admin] PATH_INFO: `/v3/domains` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:19.803320 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24696cc0-77cd-4edb-a968-49ee9bd2cd4d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 11:58:19.803629 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24696cc0-77cd-4edb-a968-49ee9bd2cd4d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:19.805043 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24696cc0-77cd-4edb-a968-49ee9bd2cd4d None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:19.818637 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 324/654] 10.222.0.23 () {56 vars in 1109 bytes} [Thu May 19 11:58:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:58:19.820611 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24696cc0-77cd-4edb-a968-49ee9bd2cd4d None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:19.851890 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a4643d-415b-4e98-a67a-0402c98c7790 None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:19.855393 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:58:19.887223 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:58:19.901439 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb4a1f9-49d1-4c74-9491-688fbe3c95ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:19.902237 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb4a1f9-49d1-4c74-9491-688fbe3c95ee None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:19.902524 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb4a1f9-49d1-4c74-9491-688fbe3c95ee None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:19.902932 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb4a1f9-49d1-4c74-9491-688fbe3c95ee None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:19.915747 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb4a1f9-49d1-4c74-9491-688fbe3c95ee None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:19.917258 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb4a1f9-49d1-4c74-9491-688fbe3c95ee None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:19.943988 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application [None req-ddb4a1f9-49d1-4c74-9491-688fbe3c95ee None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 11:58:19.943988 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 11:58:19.943988 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 11:58:19.943988 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 11:58:19.943988 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 11:58:19.943988 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 11:58:19.943988 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 11:58:19.943988 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 11:58:19.943988 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 11:58:19.943988 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 11:58:19.943988 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 11:58:19.943988 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 11:58:19.943988 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 11:58:19.943988 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 11:58:19.943988 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 11:58:19.944958 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 11:58:19.944958 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 11:58:19.944958 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 11:58:19.944958 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 11:58:19.944958 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 11:58:19.944958 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 11:58:19.944958 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.get_or_create( May 19 11:58:19.944958 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 11:58:19.944958 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application with Lock( May 19 11:58:19.944958 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 11:58:19.944958 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._enter() May 19 11:58:19.944958 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 11:58:19.944958 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 11:58:19.944958 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 11:58:19.944958 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.creator() May 19 11:58:19.944958 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 11:58:19.944958 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application created_value = creator( May 19 11:58:19.946094 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 11:58:19.946094 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 11:58:19.946094 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 11:58:19.946094 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application May 19 11:58:19.950233 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 331/655] 10.222.0.23 () {56 vars in 1108 bytes} [Thu May 19 11:58:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 11:58:19.952934 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb4a1f9-49d1-4c74-9491-688fbe3c95ee None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:19.961947 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24696cc0-77cd-4edb-a968-49ee9bd2cd4d None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:20.011451 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463844f1-42f5-4b74-a6c5-8a46393b5a5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:20.012534 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463844f1-42f5-4b74-a6c5-8a46393b5a5a None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:20.012998 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463844f1-42f5-4b74-a6c5-8a46393b5a5a None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:20.013446 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463844f1-42f5-4b74-a6c5-8a46393b5a5a None admin] PATH_INFO: `/v3/projects` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:20.014948 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463844f1-42f5-4b74-a6c5-8a46393b5a5a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 11:58:20.015458 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463844f1-42f5-4b74-a6c5-8a46393b5a5a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:20.018818 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463844f1-42f5-4b74-a6c5-8a46393b5a5a None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:20.044131 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 325/656] 10.222.0.23 () {56 vars in 1148 bytes} [Thu May 19 11:58:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:58:20.044131 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463844f1-42f5-4b74-a6c5-8a46393b5a5a None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:20.113029 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb4a1f9-49d1-4c74-9491-688fbe3c95ee None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:20.137490 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:58:20.150828 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0adaef8-1729-4d39-964b-38439a4acb98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:20.151566 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0adaef8-1729-4d39-964b-38439a4acb98 None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:20.151830 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0adaef8-1729-4d39-964b-38439a4acb98 None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:20.152100 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0adaef8-1729-4d39-964b-38439a4acb98 None admin] PATH_INFO: `/v3/users` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:20.152943 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0adaef8-1729-4d39-964b-38439a4acb98 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 11:58:20.153258 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0adaef8-1729-4d39-964b-38439a4acb98 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:20.153723 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463844f1-42f5-4b74-a6c5-8a46393b5a5a None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:20.156060 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0adaef8-1729-4d39-964b-38439a4acb98 None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:20.175429 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 332/657] 10.222.0.23 () {56 vars in 1137 bytes} [Thu May 19 11:58:19 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:58:20.176136 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0adaef8-1729-4d39-964b-38439a4acb98 None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:20.197378 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e140a5-429e-49d9-96de-65fc6909bd85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:20.198562 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e140a5-429e-49d9-96de-65fc6909bd85 None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:20.199080 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e140a5-429e-49d9-96de-65fc6909bd85 None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:20.199575 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e140a5-429e-49d9-96de-65fc6909bd85 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:20.206938 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e140a5-429e-49d9-96de-65fc6909bd85 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:20.208591 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e140a5-429e-49d9-96de-65fc6909bd85 None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:20.213443 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:58:20.224644 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application [None req-a1e140a5-429e-49d9-96de-65fc6909bd85 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 11:58:20.224644 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 11:58:20.224644 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 11:58:20.224644 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 11:58:20.224644 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 11:58:20.224644 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 11:58:20.224644 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 11:58:20.224644 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 11:58:20.224644 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 11:58:20.224644 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 11:58:20.224644 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 11:58:20.224644 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 11:58:20.224644 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 11:58:20.224644 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 11:58:20.224644 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 11:58:20.231986 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 11:58:20.231986 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 11:58:20.231986 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 11:58:20.231986 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 11:58:20.231986 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 11:58:20.231986 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 11:58:20.231986 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.get_or_create( May 19 11:58:20.231986 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 11:58:20.231986 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application with Lock( May 19 11:58:20.231986 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 11:58:20.231986 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._enter() May 19 11:58:20.231986 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 11:58:20.231986 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 11:58:20.231986 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 11:58:20.231986 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.creator() May 19 11:58:20.231986 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 11:58:20.231986 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application created_value = creator( May 19 11:58:20.233712 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 11:58:20.233712 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 11:58:20.233712 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 11:58:20.233712 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 11:58:20.233712 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 11:58:20.233712 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 11:58:20.233712 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 11:58:20.233712 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application May 19 11:58:20.233712 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 326/658] 10.222.0.23 () {56 vars in 1111 bytes} [Thu May 19 11:58:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 11:58:20.233712 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e140a5-429e-49d9-96de-65fc6909bd85 None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:20.293638 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0adaef8-1729-4d39-964b-38439a4acb98 None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:20.338025 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725dc165-9b93-4fec-8303-cceb3a773480 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:20.338702 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725dc165-9b93-4fec-8303-cceb3a773480 None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:20.338988 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725dc165-9b93-4fec-8303-cceb3a773480 None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:20.339266 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725dc165-9b93-4fec-8303-cceb3a773480 None admin] PATH_INFO: `/v3/domains` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:20.340145 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725dc165-9b93-4fec-8303-cceb3a773480 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 11:58:20.340577 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725dc165-9b93-4fec-8303-cceb3a773480 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:20.342158 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725dc165-9b93-4fec-8303-cceb3a773480 None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:20.343509 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e140a5-429e-49d9-96de-65fc6909bd85 None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:20.355582 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 333/659] 10.222.0.23 () {56 vars in 1109 bytes} [Thu May 19 11:58:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:58:20.356493 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725dc165-9b93-4fec-8303-cceb3a773480 None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:20.383266 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5733d85c-5c85-40ca-9968-c83308ce267f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:20.383878 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5733d85c-5c85-40ca-9968-c83308ce267f None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:20.384165 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5733d85c-5c85-40ca-9968-c83308ce267f None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:20.384444 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5733d85c-5c85-40ca-9968-c83308ce267f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:20.385397 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5733d85c-5c85-40ca-9968-c83308ce267f None admin] RBAC: Adding query filter params (role.id=7c59061ba7064a0db6032433fb0eec6b, scope.project.id=d70d25bf69314820b8aaecb65576978a, user.id=34a98d75d84b4c67981377e0f26eca62) {{(pid=80539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 11:58:20.385714 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5733d85c-5c85-40ca-9968-c83308ce267f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:20.387293 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5733d85c-5c85-40ca-9968-c83308ce267f None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:20.400011 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 327/660] 10.222.0.23 () {56 vars in 1374 bytes} [Thu May 19 11:58:20 2022] GET /identity/v3/role_assignments?user.id=34a98d75d84b4c67981377e0f26eca62&scope.project.id=d70d25bf69314820b8aaecb65576978a&role.id=7c59061ba7064a0db6032433fb0eec6b => generated 608 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:58:20.400011 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5733d85c-5c85-40ca-9968-c83308ce267f None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:20.413856 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:58:20.484117 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:58:20.491895 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725dc165-9b93-4fec-8303-cceb3a773480 None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:20.531706 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a16879-cb3f-473c-bb78-c5fef0a03333 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:20.532432 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a16879-cb3f-473c-bb78-c5fef0a03333 None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:20.532721 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a16879-cb3f-473c-bb78-c5fef0a03333 None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:20.533003 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a16879-cb3f-473c-bb78-c5fef0a03333 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:20.542468 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5733d85c-5c85-40ca-9968-c83308ce267f None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:20.583116 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8034303-52ae-4e74-9606-c3e79a341372 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:20.583777 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8034303-52ae-4e74-9606-c3e79a341372 None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:20.584101 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8034303-52ae-4e74-9606-c3e79a341372 None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:20.584381 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8034303-52ae-4e74-9606-c3e79a341372 None admin] PATH_INFO: `/v3/projects` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:20.585189 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8034303-52ae-4e74-9606-c3e79a341372 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 11:58:20.585504 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8034303-52ae-4e74-9606-c3e79a341372 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:20.586979 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8034303-52ae-4e74-9606-c3e79a341372 None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:20.600603 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 328/661] 10.222.0.23 () {56 vars in 1148 bytes} [Thu May 19 11:58:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:58:20.600830 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8034303-52ae-4e74-9606-c3e79a341372 None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:20.626134 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a16879-cb3f-473c-bb78-c5fef0a03333 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:20.628307 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a16879-cb3f-473c-bb78-c5fef0a03333 None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:20.692449 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:58:20.726378 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8034303-52ae-4e74-9606-c3e79a341372 None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:20.739319 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:58:20.751996 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 334/662] 10.222.0.23 () {58 vars in 1282 bytes} [Thu May 19 11:58:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 11:58:20.753454 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3874578b-2067-4ed1-aff8-08459376feaa None None] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:20.753752 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3874578b-2067-4ed1-aff8-08459376feaa None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:20.754040 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3874578b-2067-4ed1-aff8-08459376feaa None None] PATH_INFO: `/` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:20.755529 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 335/663] 10.222.0.23 () {54 vars in 926 bytes} [Thu May 19 11:58:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 11:58:20.760496 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3874578b-2067-4ed1-aff8-08459376feaa None None] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:20.767382 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c9f5e3-45c4-4e8a-98f3-c241bc66d782 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:20.768274 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c9f5e3-45c4-4e8a-98f3-c241bc66d782 None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:20.768550 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c9f5e3-45c4-4e8a-98f3-c241bc66d782 None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:20.768777 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c9f5e3-45c4-4e8a-98f3-c241bc66d782 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:20.777379 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c9f5e3-45c4-4e8a-98f3-c241bc66d782 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:20.778831 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c9f5e3-45c4-4e8a-98f3-c241bc66d782 None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:20.804049 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application [None req-59c9f5e3-45c4-4e8a-98f3-c241bc66d782 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. May 19 11:58:20.804049 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 11:58:20.804049 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 11:58:20.804049 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 11:58:20.804049 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 11:58:20.804049 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 11:58:20.804049 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 11:58:20.804049 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 11:58:20.804049 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 11:58:20.804049 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 11:58:20.804049 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 11:58:20.804049 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 11:58:20.804049 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 11:58:20.804049 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 11:58:20.804049 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 11:58:20.804049 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 11:58:20.805088 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 11:58:20.805088 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 11:58:20.805088 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 11:58:20.805088 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 11:58:20.805088 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 11:58:20.805088 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 11:58:20.805088 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 11:58:20.805088 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 11:58:20.805088 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 11:58:20.805088 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.get_or_create( May 19 11:58:20.805088 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 11:58:20.805088 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application with Lock( May 19 11:58:20.805088 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 11:58:20.805088 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._enter() May 19 11:58:20.805088 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 11:58:20.805088 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 11:58:20.805088 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 11:58:20.805088 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.creator() May 19 11:58:20.806028 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 11:58:20.806028 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application created_value = creator( May 19 11:58:20.806028 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 11:58:20.806028 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 11:58:20.806028 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 11:58:20.806028 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 11:58:20.806028 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 11:58:20.806028 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 11:58:20.806028 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. May 19 11:58:20.806028 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application May 19 11:58:20.809950 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 329/664] 10.222.0.23 () {56 vars in 1102 bytes} [Thu May 19 11:58:20 2022] GET /identity/v3/users/neutron => generated 85 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 11:58:20.810690 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c9f5e3-45c4-4e8a-98f3-c241bc66d782 None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:20.896517 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3874578b-2067-4ed1-aff8-08459376feaa None None] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:20.927540 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c9f5e3-45c4-4e8a-98f3-c241bc66d782 None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:20.937024 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5000a3-2da7-4fed-9fb3-d523382d76f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:20.937693 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5000a3-2da7-4fed-9fb3-d523382d76f5 None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:20.937986 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5000a3-2da7-4fed-9fb3-d523382d76f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:20.938273 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5000a3-2da7-4fed-9fb3-d523382d76f5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:20.939112 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5000a3-2da7-4fed-9fb3-d523382d76f5 None admin] RBAC: Adding query filter params (role.id=7c59061ba7064a0db6032433fb0eec6b, scope.project.id=d70d25bf69314820b8aaecb65576978a, user.id=8800254c0e784e42a791fb9abc274d79) {{(pid=80540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 11:58:20.939429 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5000a3-2da7-4fed-9fb3-d523382d76f5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:20.940925 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5000a3-2da7-4fed-9fb3-d523382d76f5 None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:20.961190 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:58:20.963992 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 336/665] 10.222.0.23 () {56 vars in 1374 bytes} [Thu May 19 11:58:20 2022] GET /identity/v3/role_assignments?user.id=8800254c0e784e42a791fb9abc274d79&scope.project.id=d70d25bf69314820b8aaecb65576978a&role.id=7c59061ba7064a0db6032433fb0eec6b => generated 608 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:58:20.965505 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e9e738-5587-410e-b244-51518b3f486b None None] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:20.965801 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e9e738-5587-410e-b244-51518b3f486b None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:20.966084 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e9e738-5587-410e-b244-51518b3f486b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:20.976882 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf65010a-a752-410b-bdd5-5216526aac41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:20.977713 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf65010a-a752-410b-bdd5-5216526aac41 None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:20.978000 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf65010a-a752-410b-bdd5-5216526aac41 None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:20.978294 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf65010a-a752-410b-bdd5-5216526aac41 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:20.993893 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf65010a-a752-410b-bdd5-5216526aac41 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:20.994435 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.auth.core [None req-83e9e738-5587-410e-b244-51518b3f486b None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 11:58:20.995397 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf65010a-a752-410b-bdd5-5216526aac41 None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:20.995665 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-83e9e738-5587-410e-b244-51518b3f486b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 11:58:21.019221 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application [None req-cf65010a-a752-410b-bdd5-5216526aac41 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 11:58:21.019221 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 11:58:21.019221 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 11:58:21.019221 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 11:58:21.019221 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 11:58:21.019221 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 11:58:21.019221 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 11:58:21.019221 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 11:58:21.019221 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 11:58:21.019221 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 11:58:21.019221 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 11:58:21.019221 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 11:58:21.019221 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 11:58:21.019221 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 11:58:21.019221 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 11:58:21.020263 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 11:58:21.020263 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 11:58:21.020263 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 11:58:21.020263 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 11:58:21.020263 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 11:58:21.020263 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 11:58:21.020263 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.get_or_create( May 19 11:58:21.020263 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 11:58:21.020263 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application with Lock( May 19 11:58:21.020263 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 11:58:21.020263 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._enter() May 19 11:58:21.020263 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 11:58:21.020263 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 11:58:21.020263 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 11:58:21.020263 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.creator() May 19 11:58:21.020263 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 11:58:21.020263 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application created_value = creator( May 19 11:58:21.021140 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 11:58:21.021140 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 11:58:21.021140 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 11:58:21.021140 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application May 19 11:58:21.021786 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 330/666] 10.222.0.23 () {56 vars in 1108 bytes} [Thu May 19 11:58:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 11:58:21.022537 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf65010a-a752-410b-bdd5-5216526aac41 None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:21.069162 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:58:21.085870 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 337/667] 10.222.0.23 () {58 vars in 1023 bytes} [Thu May 19 11:58:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 11:58:21.086504 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e9e738-5587-410e-b244-51518b3f486b None None] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:21.172917 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf65010a-a752-410b-bdd5-5216526aac41 None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:21.206531 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e9e738-5587-410e-b244-51518b3f486b None None] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:21.236975 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd38e31-c085-4641-8b5d-fcba93eb7a5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:21.237652 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd38e31-c085-4641-8b5d-fcba93eb7a5f None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:21.237931 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd38e31-c085-4641-8b5d-fcba93eb7a5f None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:21.238204 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd38e31-c085-4641-8b5d-fcba93eb7a5f None admin] PATH_INFO: `/v3/users` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:21.239049 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd38e31-c085-4641-8b5d-fcba93eb7a5f None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 11:58:21.239362 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd38e31-c085-4641-8b5d-fcba93eb7a5f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:21.241642 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd38e31-c085-4641-8b5d-fcba93eb7a5f None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:21.261369 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 331/668] 10.222.0.23 () {56 vars in 1139 bytes} [Thu May 19 11:58:21 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:58:21.263725 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd38e31-c085-4641-8b5d-fcba93eb7a5f None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:21.264636 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f5b52a-086e-401b-a0b7-5d3b3650346f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:21.265528 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f5b52a-086e-401b-a0b7-5d3b3650346f None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:21.265982 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f5b52a-086e-401b-a0b7-5d3b3650346f None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:21.266399 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f5b52a-086e-401b-a0b7-5d3b3650346f None admin] PATH_INFO: `/v3/domains` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:21.268471 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f5b52a-086e-401b-a0b7-5d3b3650346f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 11:58:21.268848 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f5b52a-086e-401b-a0b7-5d3b3650346f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:21.270278 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f5b52a-086e-401b-a0b7-5d3b3650346f None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:21.286519 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 338/669] 10.222.0.23 () {56 vars in 1109 bytes} [Thu May 19 11:58:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:58:21.288163 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f5b52a-086e-401b-a0b7-5d3b3650346f None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:21.314898 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:58:21.352411 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:58:21.404014 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd38e31-c085-4641-8b5d-fcba93eb7a5f None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:21.411596 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f5b52a-086e-401b-a0b7-5d3b3650346f None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:21.451956 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428d857b-27c4-4f3e-b518-5b308c137994 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:21.452858 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428d857b-27c4-4f3e-b518-5b308c137994 None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:21.453143 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428d857b-27c4-4f3e-b518-5b308c137994 None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:21.453257 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ffd5f4-93a5-48c0-9a85-f9c9d77c2d4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:21.453413 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428d857b-27c4-4f3e-b518-5b308c137994 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:21.454264 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ffd5f4-93a5-48c0-9a85-f9c9d77c2d4f None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:21.454646 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ffd5f4-93a5-48c0-9a85-f9c9d77c2d4f None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:21.455022 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ffd5f4-93a5-48c0-9a85-f9c9d77c2d4f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:21.471646 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428d857b-27c4-4f3e-b518-5b308c137994 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 11:58:21.482926 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application [None req-428d857b-27c4-4f3e-b518-5b308c137994 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 11:58:21.482926 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 11:58:21.482926 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 11:58:21.482926 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 11:58:21.482926 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 11:58:21.482926 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 11:58:21.482926 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 11:58:21.482926 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 11:58:21.482926 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 11:58:21.482926 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 11:58:21.482926 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 11:58:21.482926 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 11:58:21.482926 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 11:58:21.482926 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 11:58:21.482926 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 11:58:21.482926 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 11:58:21.484189 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 11:58:21.484189 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 11:58:21.484189 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 11:58:21.484189 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 11:58:21.484189 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 11:58:21.484189 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 11:58:21.484189 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 11:58:21.484189 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 11:58:21.484189 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 11:58:21.484189 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.get_or_create( May 19 11:58:21.484189 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 11:58:21.484189 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application with Lock( May 19 11:58:21.484189 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 11:58:21.484189 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._enter() May 19 11:58:21.484189 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 11:58:21.484189 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 11:58:21.485026 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 11:58:21.485026 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.creator() May 19 11:58:21.485026 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 11:58:21.485026 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application created_value = creator( May 19 11:58:21.485026 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 11:58:21.485026 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 11:58:21.485026 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 11:58:21.485026 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 11:58:21.485026 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 11:58:21.485026 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 11:58:21.485026 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 11:58:21.485026 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application May 19 11:58:21.485026 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 332/670] 10.222.0.23 () {56 vars in 1102 bytes} [Thu May 19 11:58:21 2022] GET /identity/v3/roles/service => generated 85 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 11:58:21.485943 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428d857b-27c4-4f3e-b518-5b308c137994 None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:21.538763 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ffd5f4-93a5-48c0-9a85-f9c9d77c2d4f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:21.547468 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ffd5f4-93a5-48c0-9a85-f9c9d77c2d4f None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:21.602874 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:58:21.615717 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:58:21.619138 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428d857b-27c4-4f3e-b518-5b308c137994 None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:21.668024 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 339/671] 10.222.0.23 () {58 vars in 1282 bytes} [Thu May 19 11:58:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 11:58:21.669321 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ffd5f4-93a5-48c0-9a85-f9c9d77c2d4f None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:21.673220 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbcd49c8-a9c6-40ba-8a54-de8f19b23371 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:21.675009 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcd49c8-a9c6-40ba-8a54-de8f19b23371 None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:21.675408 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcd49c8-a9c6-40ba-8a54-de8f19b23371 None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:21.675694 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcd49c8-a9c6-40ba-8a54-de8f19b23371 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:21.686436 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbcd49c8-a9c6-40ba-8a54-de8f19b23371 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:21.688135 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbcd49c8-a9c6-40ba-8a54-de8f19b23371 None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:21.710403 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application [None req-cbcd49c8-a9c6-40ba-8a54-de8f19b23371 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 11:58:21.710403 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 11:58:21.710403 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 11:58:21.710403 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 11:58:21.710403 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 11:58:21.710403 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 11:58:21.710403 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 11:58:21.710403 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 11:58:21.710403 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 11:58:21.710403 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 11:58:21.710403 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 11:58:21.710403 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 11:58:21.710403 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 11:58:21.710403 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 11:58:21.710403 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 11:58:21.718343 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 11:58:21.718343 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 11:58:21.718343 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 11:58:21.718343 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 11:58:21.718343 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 11:58:21.718343 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 11:58:21.718343 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.get_or_create( May 19 11:58:21.718343 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 11:58:21.718343 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application with Lock( May 19 11:58:21.718343 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 11:58:21.718343 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._enter() May 19 11:58:21.718343 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 11:58:21.718343 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 11:58:21.718343 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 11:58:21.718343 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.creator() May 19 11:58:21.718343 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 11:58:21.718343 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application created_value = creator( May 19 11:58:21.719294 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 11:58:21.719294 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 11:58:21.719294 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 11:58:21.719294 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 11:58:21.719294 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 11:58:21.719294 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 11:58:21.719294 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 11:58:21.719294 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application May 19 11:58:21.719294 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 333/672] 10.222.0.23 () {56 vars in 1111 bytes} [Thu May 19 11:58:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 11:58:21.719294 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbcd49c8-a9c6-40ba-8a54-de8f19b23371 None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:21.797548 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ffd5f4-93a5-48c0-9a85-f9c9d77c2d4f None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:21.842315 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3b6e26-c040-4935-b8b6-f916040fb193 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:21.842835 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3b6e26-c040-4935-b8b6-f916040fb193 None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:21.843313 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3b6e26-c040-4935-b8b6-f916040fb193 None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:21.843607 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3b6e26-c040-4935-b8b6-f916040fb193 None admin] PATH_INFO: `/v3/roles` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:21.844422 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3b6e26-c040-4935-b8b6-f916040fb193 None admin] RBAC: Adding query filter params (name=service) {{(pid=80540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 11:58:21.844739 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3b6e26-c040-4935-b8b6-f916040fb193 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:21.846146 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3b6e26-c040-4935-b8b6-f916040fb193 None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:21.859534 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 340/673] 10.222.0.23 () {56 vars in 1103 bytes} [Thu May 19 11:58:21 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:58:21.860494 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3b6e26-c040-4935-b8b6-f916040fb193 None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:21.873882 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbcd49c8-a9c6-40ba-8a54-de8f19b23371 None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:21.884854 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:58:21.916723 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3666e6d-59a8-4048-b0f7-c45a4f6e84a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:21.917521 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3666e6d-59a8-4048-b0f7-c45a4f6e84a7 None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:21.918499 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3666e6d-59a8-4048-b0f7-c45a4f6e84a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:21.918499 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3666e6d-59a8-4048-b0f7-c45a4f6e84a7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:21.933926 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3666e6d-59a8-4048-b0f7-c45a4f6e84a7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:21.936219 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3666e6d-59a8-4048-b0f7-c45a4f6e84a7 None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:21.960558 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application [None req-c3666e6d-59a8-4048-b0f7-c45a4f6e84a7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 11:58:21.960558 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 11:58:21.960558 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 11:58:21.960558 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 11:58:21.960558 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 11:58:21.960558 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 11:58:21.960558 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 11:58:21.960558 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 11:58:21.960558 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 11:58:21.960558 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 11:58:21.960558 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 11:58:21.960558 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 11:58:21.960558 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 11:58:21.960558 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 11:58:21.960558 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 11:58:21.961643 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 11:58:21.961643 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 11:58:21.961643 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 11:58:21.961643 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 11:58:21.961643 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 11:58:21.961643 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 11:58:21.961643 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.get_or_create( May 19 11:58:21.961643 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 11:58:21.961643 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application with Lock( May 19 11:58:21.961643 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 11:58:21.961643 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._enter() May 19 11:58:21.961643 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 11:58:21.961643 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 11:58:21.961643 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 11:58:21.961643 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.creator() May 19 11:58:21.961643 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 11:58:21.961643 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application created_value = creator( May 19 11:58:21.962602 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 11:58:21.962602 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 11:58:21.962602 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 11:58:21.962602 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application May 19 11:58:21.962602 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 334/674] 10.222.0.23 () {56 vars in 1108 bytes} [Thu May 19 11:58:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 11:58:21.962880 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3666e6d-59a8-4048-b0f7-c45a4f6e84a7 None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:21.988568 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:58:22.003486 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3b6e26-c040-4935-b8b6-f916040fb193 None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:22.054832 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f974c7fd-7923-4301-b880-b6c92bc3e9ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:22.055736 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f974c7fd-7923-4301-b880-b6c92bc3e9ed None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:22.056199 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f974c7fd-7923-4301-b880-b6c92bc3e9ed None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:22.056902 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f974c7fd-7923-4301-b880-b6c92bc3e9ed None admin] PATH_INFO: `/v3/projects` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:22.058422 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f974c7fd-7923-4301-b880-b6c92bc3e9ed None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 11:58:22.058910 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f974c7fd-7923-4301-b880-b6c92bc3e9ed None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:22.061607 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f974c7fd-7923-4301-b880-b6c92bc3e9ed None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:22.079869 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 341/675] 10.222.0.23 () {56 vars in 1148 bytes} [Thu May 19 11:58:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:58:22.081031 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f974c7fd-7923-4301-b880-b6c92bc3e9ed None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:22.095750 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3666e6d-59a8-4048-b0f7-c45a4f6e84a7 None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:22.134972 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf91d47-19be-4f38-a43c-7c4842068401 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:22.135996 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf91d47-19be-4f38-a43c-7c4842068401 None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:22.136191 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf91d47-19be-4f38-a43c-7c4842068401 None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:22.136472 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf91d47-19be-4f38-a43c-7c4842068401 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:22.148865 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf91d47-19be-4f38-a43c-7c4842068401 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:22.150442 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf91d47-19be-4f38-a43c-7c4842068401 None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:22.165369 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:58:22.171615 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application [None req-daf91d47-19be-4f38-a43c-7c4842068401 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 11:58:22.171615 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 11:58:22.171615 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 11:58:22.171615 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 11:58:22.171615 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 11:58:22.171615 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 11:58:22.171615 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 11:58:22.171615 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 11:58:22.171615 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 11:58:22.171615 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 11:58:22.171615 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 11:58:22.171615 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 11:58:22.171615 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 11:58:22.171615 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 11:58:22.171615 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 11:58:22.172709 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 11:58:22.172709 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 11:58:22.172709 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 11:58:22.172709 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 11:58:22.172709 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 11:58:22.172709 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 11:58:22.172709 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.get_or_create( May 19 11:58:22.172709 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 11:58:22.172709 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application with Lock( May 19 11:58:22.172709 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 11:58:22.172709 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._enter() May 19 11:58:22.172709 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 11:58:22.172709 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 11:58:22.172709 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 11:58:22.172709 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.creator() May 19 11:58:22.172709 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 11:58:22.172709 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application created_value = creator( May 19 11:58:22.173726 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 11:58:22.173726 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 11:58:22.173726 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 11:58:22.173726 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application May 19 11:58:22.177624 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 335/676] 10.222.0.23 () {56 vars in 1108 bytes} [Thu May 19 11:58:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 11:58:22.184913 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf91d47-19be-4f38-a43c-7c4842068401 None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:22.203431 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f974c7fd-7923-4301-b880-b6c92bc3e9ed None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:22.241861 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0c2c39-40b5-4efb-8df2-9dc0338865f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:22.242776 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0c2c39-40b5-4efb-8df2-9dc0338865f4 None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:22.243233 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0c2c39-40b5-4efb-8df2-9dc0338865f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:22.243658 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0c2c39-40b5-4efb-8df2-9dc0338865f4 None admin] PATH_INFO: `/v3/domains` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:22.244536 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0c2c39-40b5-4efb-8df2-9dc0338865f4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 11:58:22.245203 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0c2c39-40b5-4efb-8df2-9dc0338865f4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:22.246799 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0c2c39-40b5-4efb-8df2-9dc0338865f4 None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:22.249640 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:58:22.264560 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 342/677] 10.222.0.23 () {56 vars in 1109 bytes} [Thu May 19 11:58:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:58:22.264560 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0c2c39-40b5-4efb-8df2-9dc0338865f4 None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:22.300629 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf91d47-19be-4f38-a43c-7c4842068401 None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:22.340622 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad464275-f915-4db4-9c02-ce24e248e3b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:22.341819 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad464275-f915-4db4-9c02-ce24e248e3b1 None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:22.342821 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad464275-f915-4db4-9c02-ce24e248e3b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:22.343335 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad464275-f915-4db4-9c02-ce24e248e3b1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:22.344397 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad464275-f915-4db4-9c02-ce24e248e3b1 None admin] RBAC: Adding query filter params (role.id=7c59061ba7064a0db6032433fb0eec6b, scope.project.id=d70d25bf69314820b8aaecb65576978a, user.id=21697802f2be42f9baa8e07490d5ebdd) {{(pid=80539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 11:58:22.344867 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad464275-f915-4db4-9c02-ce24e248e3b1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:22.346602 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad464275-f915-4db4-9c02-ce24e248e3b1 None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:22.354042 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 336/678] 10.222.0.23 () {56 vars in 1374 bytes} [Thu May 19 11:58:22 2022] GET /identity/v3/role_assignments?user.id=21697802f2be42f9baa8e07490d5ebdd&scope.project.id=d70d25bf69314820b8aaecb65576978a&role.id=7c59061ba7064a0db6032433fb0eec6b => generated 608 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:58:22.355214 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad464275-f915-4db4-9c02-ce24e248e3b1 None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:22.374647 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0c2c39-40b5-4efb-8df2-9dc0338865f4 None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:22.407692 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:58:22.413659 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4710c589-9ee0-447c-b4fa-d36a96d60ffe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:22.414726 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4710c589-9ee0-447c-b4fa-d36a96d60ffe None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:22.414850 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4710c589-9ee0-447c-b4fa-d36a96d60ffe None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:22.415062 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4710c589-9ee0-447c-b4fa-d36a96d60ffe None admin] PATH_INFO: `/v3/domains` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:22.415830 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4710c589-9ee0-447c-b4fa-d36a96d60ffe None admin] RBAC: Adding query filter params (name=Default) {{(pid=80540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 11:58:22.416230 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4710c589-9ee0-447c-b4fa-d36a96d60ffe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:22.417583 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4710c589-9ee0-447c-b4fa-d36a96d60ffe None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:22.431025 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 343/679] 10.222.0.23 () {56 vars in 1109 bytes} [Thu May 19 11:58:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:58:22.437288 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4710c589-9ee0-447c-b4fa-d36a96d60ffe None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:22.480229 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad464275-f915-4db4-9c02-ce24e248e3b1 None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:22.512937 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:58:22.523721 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b97729e-5be8-4c5e-8030-df42b9b73760 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:22.524836 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b97729e-5be8-4c5e-8030-df42b9b73760 None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:22.525276 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b97729e-5be8-4c5e-8030-df42b9b73760 None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:22.525707 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b97729e-5be8-4c5e-8030-df42b9b73760 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:22.534575 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b97729e-5be8-4c5e-8030-df42b9b73760 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:22.536508 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b97729e-5be8-4c5e-8030-df42b9b73760 None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:22.553876 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application [None req-0b97729e-5be8-4c5e-8030-df42b9b73760 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 11:58:22.553876 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 11:58:22.553876 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 11:58:22.553876 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 11:58:22.553876 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 11:58:22.553876 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 11:58:22.553876 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 11:58:22.553876 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 11:58:22.553876 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 11:58:22.553876 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 11:58:22.553876 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 11:58:22.553876 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 11:58:22.553876 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 11:58:22.553876 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 11:58:22.553876 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 11:58:22.555006 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 11:58:22.555006 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 11:58:22.555006 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 11:58:22.555006 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 11:58:22.555006 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 11:58:22.555006 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 11:58:22.555006 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.get_or_create( May 19 11:58:22.555006 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 11:58:22.555006 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application with Lock( May 19 11:58:22.555006 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 11:58:22.555006 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._enter() May 19 11:58:22.555006 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 11:58:22.555006 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 11:58:22.555006 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 11:58:22.555006 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.creator() May 19 11:58:22.555006 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 11:58:22.555006 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application created_value = creator( May 19 11:58:22.556087 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 11:58:22.556087 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 11:58:22.556087 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 11:58:22.556087 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 11:58:22.556087 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 11:58:22.556087 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 11:58:22.556087 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 11:58:22.556087 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application May 19 11:58:22.556524 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4710c589-9ee0-447c-b4fa-d36a96d60ffe None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:22.558718 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 337/680] 10.222.0.23 () {56 vars in 1111 bytes} [Thu May 19 11:58:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 11:58:22.564991 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b97729e-5be8-4c5e-8030-df42b9b73760 None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:22.593426 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f756ca73-7a09-44a5-9985-06c3dfee10b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:22.594243 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f756ca73-7a09-44a5-9985-06c3dfee10b4 None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:22.594531 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f756ca73-7a09-44a5-9985-06c3dfee10b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:22.594806 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f756ca73-7a09-44a5-9985-06c3dfee10b4 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:22.603385 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f756ca73-7a09-44a5-9985-06c3dfee10b4 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:22.604924 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f756ca73-7a09-44a5-9985-06c3dfee10b4 None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:22.620792 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application [None req-f756ca73-7a09-44a5-9985-06c3dfee10b4 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 19 11:58:22.620792 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 11:58:22.620792 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 11:58:22.620792 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 11:58:22.620792 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 11:58:22.620792 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 11:58:22.620792 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 11:58:22.620792 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 11:58:22.620792 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 11:58:22.620792 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 11:58:22.620792 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 11:58:22.620792 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 11:58:22.620792 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 11:58:22.620792 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 11:58:22.620792 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 11:58:22.620792 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 11:58:22.621894 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 11:58:22.621894 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 11:58:22.621894 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 11:58:22.621894 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 11:58:22.621894 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 11:58:22.621894 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 11:58:22.621894 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 11:58:22.621894 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 11:58:22.621894 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 11:58:22.621894 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.get_or_create( May 19 11:58:22.621894 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 11:58:22.621894 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application with Lock( May 19 11:58:22.621894 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 11:58:22.621894 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._enter() May 19 11:58:22.621894 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 11:58:22.621894 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 11:58:22.621894 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 11:58:22.621894 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.creator() May 19 11:58:22.622817 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 11:58:22.622817 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application created_value = creator( May 19 11:58:22.622817 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 11:58:22.622817 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 11:58:22.622817 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 11:58:22.622817 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 11:58:22.622817 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 11:58:22.622817 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 11:58:22.622817 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 19 11:58:22.622817 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application May 19 11:58:22.622817 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 344/681] 10.222.0.23 () {56 vars in 1096 bytes} [Thu May 19 11:58:22 2022] GET /identity/v3/users/swift => generated 83 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 11:58:22.629027 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f756ca73-7a09-44a5-9985-06c3dfee10b4 None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:22.683471 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:58:22.691204 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b97729e-5be8-4c5e-8030-df42b9b73760 None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:22.731030 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b0a6be-9a69-44a5-b89d-48b626b3ce44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:22.732237 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b0a6be-9a69-44a5-b89d-48b626b3ce44 None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:22.733540 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b0a6be-9a69-44a5-b89d-48b626b3ce44 None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:22.734046 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b0a6be-9a69-44a5-b89d-48b626b3ce44 None admin] PATH_INFO: `/v3/projects` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:22.735067 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b0a6be-9a69-44a5-b89d-48b626b3ce44 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 11:58:22.735693 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b0a6be-9a69-44a5-b89d-48b626b3ce44 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:22.737427 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b0a6be-9a69-44a5-b89d-48b626b3ce44 None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:22.739402 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f756ca73-7a09-44a5-9985-06c3dfee10b4 None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:22.752547 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 338/682] 10.222.0.23 () {56 vars in 1148 bytes} [Thu May 19 11:58:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 11:58:22.758670 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b0a6be-9a69-44a5-b89d-48b626b3ce44 None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:22.775349 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b43793c-525b-4cba-9eba-ad6165d9188a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:22.776507 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b43793c-525b-4cba-9eba-ad6165d9188a None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:22.776507 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b43793c-525b-4cba-9eba-ad6165d9188a None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:22.776795 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b43793c-525b-4cba-9eba-ad6165d9188a None admin] PATH_INFO: `/v3/users` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:22.777648 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b43793c-525b-4cba-9eba-ad6165d9188a None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 11:58:22.777947 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b43793c-525b-4cba-9eba-ad6165d9188a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:22.779547 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b43793c-525b-4cba-9eba-ad6165d9188a None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:22.799764 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 345/683] 10.222.0.23 () {56 vars in 1135 bytes} [Thu May 19 11:58:22 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 11:58:22.805259 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b43793c-525b-4cba-9eba-ad6165d9188a None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:22.816344 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:58:22.882391 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b0a6be-9a69-44a5-b89d-48b626b3ce44 None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:22.922467 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b43793c-525b-4cba-9eba-ad6165d9188a None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:22.924244 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173df15a-af1e-48ea-a380-464fbab3121a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:22.925186 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173df15a-af1e-48ea-a380-464fbab3121a None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:22.925628 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173df15a-af1e-48ea-a380-464fbab3121a None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:22.926058 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173df15a-af1e-48ea-a380-464fbab3121a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:22.927056 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173df15a-af1e-48ea-a380-464fbab3121a None admin] RBAC: Adding query filter params (role.id=7c59061ba7064a0db6032433fb0eec6b, scope.project.id=d70d25bf69314820b8aaecb65576978a, user.id=1a00b3fa7497456895e5e0bf105c4fc6) {{(pid=80539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 11:58:22.927540 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173df15a-af1e-48ea-a380-464fbab3121a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:22.929182 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173df15a-af1e-48ea-a380-464fbab3121a None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:22.933332 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:58:22.938498 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 339/684] 10.222.0.23 () {56 vars in 1374 bytes} [Thu May 19 11:58:22 2022] GET /identity/v3/role_assignments?user.id=1a00b3fa7497456895e5e0bf105c4fc6&scope.project.id=d70d25bf69314820b8aaecb65576978a&role.id=7c59061ba7064a0db6032433fb0eec6b => generated 608 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 11:58:22.962610 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f49cbe-0597-4a67-8658-d6bf0890bbb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:22.963305 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f49cbe-0597-4a67-8658-d6bf0890bbb9 None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:22.963576 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f49cbe-0597-4a67-8658-d6bf0890bbb9 None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:22.963887 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f49cbe-0597-4a67-8658-d6bf0890bbb9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:23.047430 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f49cbe-0597-4a67-8658-d6bf0890bbb9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:23.049173 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f49cbe-0597-4a67-8658-d6bf0890bbb9 None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:23.083828 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd4d91d-5d41-4eda-a9e0-867c52d739a4 None None] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:23.084157 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd4d91d-5d41-4eda-a9e0-867c52d739a4 None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:23.084429 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd4d91d-5d41-4eda-a9e0-867c52d739a4 None None] PATH_INFO: `/` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:23.085965 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 340/685] 10.222.0.23 () {54 vars in 926 bytes} [Thu May 19 11:58:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 11:58:23.094012 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e42e6a-1174-4130-860b-8f6a926643e7 None None] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:23.094349 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e42e6a-1174-4130-860b-8f6a926643e7 None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:23.094579 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e42e6a-1174-4130-860b-8f6a926643e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:23.126076 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.auth.core [None req-50e42e6a-1174-4130-860b-8f6a926643e7 None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 11:58:23.128928 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-50e42e6a-1174-4130-860b-8f6a926643e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 11:58:23.177715 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 346/686] 10.222.0.23 () {58 vars in 1282 bytes} [Thu May 19 11:58:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 11:58:23.183013 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f49cbe-0597-4a67-8658-d6bf0890bbb9 None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:23.194679 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:58:23.229334 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 341/687] 10.222.0.23 () {58 vars in 1023 bytes} [Thu May 19 11:58:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 11:58:23.260767 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e42e6a-1174-4130-860b-8f6a926643e7 None None] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:23.304191 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f49cbe-0597-4a67-8658-d6bf0890bbb9 None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:23.308117 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:58:23.343426 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ede9eff-3b47-4e07-8b79-b069319f3f4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:23.344835 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ede9eff-3b47-4e07-8b79-b069319f3f4b None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:23.344835 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ede9eff-3b47-4e07-8b79-b069319f3f4b None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:23.344835 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ede9eff-3b47-4e07-8b79-b069319f3f4b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:23.355692 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ede9eff-3b47-4e07-8b79-b069319f3f4b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:23.357212 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ede9eff-3b47-4e07-8b79-b069319f3f4b None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:23.376875 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e42e6a-1174-4130-860b-8f6a926643e7 None None] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:23.377498 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application [None req-0ede9eff-3b47-4e07-8b79-b069319f3f4b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 11:58:23.377498 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 11:58:23.377498 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 11:58:23.377498 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 11:58:23.377498 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 11:58:23.377498 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 11:58:23.377498 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 11:58:23.377498 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 11:58:23.377498 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 11:58:23.377498 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 11:58:23.377498 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 11:58:23.377498 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 11:58:23.377498 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 11:58:23.377498 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 11:58:23.377498 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 11:58:23.378370 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 11:58:23.378370 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 11:58:23.378370 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 11:58:23.378370 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 11:58:23.378370 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 11:58:23.378370 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 11:58:23.378370 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.get_or_create( May 19 11:58:23.378370 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 11:58:23.378370 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application with Lock( May 19 11:58:23.378370 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 11:58:23.378370 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._enter() May 19 11:58:23.378370 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 11:58:23.378370 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 11:58:23.378370 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 11:58:23.378370 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.creator() May 19 11:58:23.378370 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 11:58:23.378370 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application created_value = creator( May 19 11:58:23.379249 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 11:58:23.379249 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 11:58:23.379249 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 11:58:23.379249 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application May 19 11:58:23.379249 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 347/688] 10.222.0.23 () {56 vars in 1108 bytes} [Thu May 19 11:58:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 11:58:23.384836 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ede9eff-3b47-4e07-8b79-b069319f3f4b None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:23.422109 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a294f8b0-b265-4f71-b04f-3de926080c61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:23.423217 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a294f8b0-b265-4f71-b04f-3de926080c61 None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:23.423707 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a294f8b0-b265-4f71-b04f-3de926080c61 None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:23.424189 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a294f8b0-b265-4f71-b04f-3de926080c61 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:23.438309 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a294f8b0-b265-4f71-b04f-3de926080c61 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 11:58:23.448067 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application [None req-a294f8b0-b265-4f71-b04f-3de926080c61 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 11:58:23.448067 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 11:58:23.448067 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 11:58:23.448067 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 11:58:23.448067 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 11:58:23.448067 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 11:58:23.448067 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 11:58:23.448067 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 11:58:23.448067 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 11:58:23.448067 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 11:58:23.448067 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 11:58:23.448067 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 11:58:23.448067 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 11:58:23.448067 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 11:58:23.448067 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 11:58:23.448067 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 11:58:23.449079 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 11:58:23.449079 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 11:58:23.449079 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 11:58:23.449079 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 11:58:23.449079 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 11:58:23.449079 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 11:58:23.449079 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 11:58:23.449079 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 11:58:23.449079 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 11:58:23.449079 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.get_or_create( May 19 11:58:23.449079 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 11:58:23.449079 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application with Lock( May 19 11:58:23.449079 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 11:58:23.449079 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._enter() May 19 11:58:23.449079 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 11:58:23.449079 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 11:58:23.449902 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 11:58:23.449902 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.creator() May 19 11:58:23.449902 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 11:58:23.449902 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application created_value = creator( May 19 11:58:23.449902 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 11:58:23.449902 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 11:58:23.449902 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 11:58:23.449902 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 11:58:23.449902 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 11:58:23.449902 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 11:58:23.449902 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 11:58:23.449902 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application May 19 11:58:23.453417 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 342/689] 10.222.0.23 () {56 vars in 1096 bytes} [Thu May 19 11:58:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 11:58:23.458608 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:58:23.459909 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a294f8b0-b265-4f71-b04f-3de926080c61 None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:23.501424 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ede9eff-3b47-4e07-8b79-b069319f3f4b None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:23.536453 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab844df8-71db-449a-b4d3-59882a77867d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:23.537781 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab844df8-71db-449a-b4d3-59882a77867d None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:23.537781 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab844df8-71db-449a-b4d3-59882a77867d None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:23.537781 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab844df8-71db-449a-b4d3-59882a77867d None admin] PATH_INFO: `/v3/domains` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:23.538390 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab844df8-71db-449a-b4d3-59882a77867d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 11:58:23.538700 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab844df8-71db-449a-b4d3-59882a77867d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:23.540376 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab844df8-71db-449a-b4d3-59882a77867d None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:23.553360 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 348/690] 10.222.0.23 () {56 vars in 1109 bytes} [Thu May 19 11:58:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:58:23.558652 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab844df8-71db-449a-b4d3-59882a77867d None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:23.665087 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab844df8-71db-449a-b4d3-59882a77867d None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:23.673753 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a294f8b0-b265-4f71-b04f-3de926080c61 None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:23.675685 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:58:23.686809 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:58:23.701734 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfed18d5-0fc0-462c-8fd0-b5ef8505fd87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:23.702546 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfed18d5-0fc0-462c-8fd0-b5ef8505fd87 None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:23.702832 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfed18d5-0fc0-462c-8fd0-b5ef8505fd87 None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:23.703109 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfed18d5-0fc0-462c-8fd0-b5ef8505fd87 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:23.710477 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfed18d5-0fc0-462c-8fd0-b5ef8505fd87 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:23.711983 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfed18d5-0fc0-462c-8fd0-b5ef8505fd87 None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:23.719713 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a79f5c1-b014-4e61-bb28-cd20cdc9a12a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:23.721180 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a79f5c1-b014-4e61-bb28-cd20cdc9a12a None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:23.721644 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a79f5c1-b014-4e61-bb28-cd20cdc9a12a None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:23.722096 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a79f5c1-b014-4e61-bb28-cd20cdc9a12a None admin] PATH_INFO: `/v3/roles` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:23.723040 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a79f5c1-b014-4e61-bb28-cd20cdc9a12a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 11:58:23.723615 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a79f5c1-b014-4e61-bb28-cd20cdc9a12a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:23.725197 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a79f5c1-b014-4e61-bb28-cd20cdc9a12a None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:23.727749 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application [None req-bfed18d5-0fc0-462c-8fd0-b5ef8505fd87 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 11:58:23.727749 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 11:58:23.727749 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 11:58:23.727749 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 11:58:23.727749 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 11:58:23.727749 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 11:58:23.727749 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 11:58:23.727749 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 11:58:23.727749 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 11:58:23.727749 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 11:58:23.727749 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 11:58:23.727749 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 11:58:23.727749 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 11:58:23.727749 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 11:58:23.727749 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 11:58:23.728709 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 11:58:23.728709 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 11:58:23.728709 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 11:58:23.728709 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 11:58:23.728709 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 11:58:23.728709 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 11:58:23.728709 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.get_or_create( May 19 11:58:23.728709 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 11:58:23.728709 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application with Lock( May 19 11:58:23.728709 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 11:58:23.728709 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._enter() May 19 11:58:23.728709 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 11:58:23.728709 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 11:58:23.728709 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 11:58:23.728709 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.creator() May 19 11:58:23.728709 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 11:58:23.728709 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application created_value = creator( May 19 11:58:23.729847 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 11:58:23.729847 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 11:58:23.729847 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 11:58:23.729847 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 11:58:23.729847 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 11:58:23.729847 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 11:58:23.729847 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 11:58:23.729847 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application May 19 11:58:23.729847 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 349/691] 10.222.0.23 () {56 vars in 1111 bytes} [Thu May 19 11:58:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 11:58:23.730554 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b34acb-0756-4beb-8dea-3278e1d8190d None None] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:23.730845 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b34acb-0756-4beb-8dea-3278e1d8190d None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:23.731127 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b34acb-0756-4beb-8dea-3278e1d8190d None None] PATH_INFO: `/` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:23.741071 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 350/692] 10.222.0.23 () {54 vars in 926 bytes} [Thu May 19 11:58:23 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 11:58:23.741071 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b34acb-0756-4beb-8dea-3278e1d8190d None None] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:23.743564 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 343/693] 10.222.0.23 () {56 vars in 1099 bytes} [Thu May 19 11:58:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:58:23.745216 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211b0fee-30e1-4155-9aab-82751cc62d5e None None] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:23.745712 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211b0fee-30e1-4155-9aab-82751cc62d5e None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:23.746275 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211b0fee-30e1-4155-9aab-82751cc62d5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:23.769025 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.auth.core [None req-211b0fee-30e1-4155-9aab-82751cc62d5e None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 11:58:23.771295 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-211b0fee-30e1-4155-9aab-82751cc62d5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 11:58:23.852993 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b34acb-0756-4beb-8dea-3278e1d8190d None None] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:23.871966 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 344/694] 10.222.0.23 () {58 vars in 1023 bytes} [Thu May 19 11:58:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 11:58:23.871966 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211b0fee-30e1-4155-9aab-82751cc62d5e None None] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:23.907224 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b5b4e8-d97d-4e8e-8daa-b3dc1f1c5e4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:23.907923 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b5b4e8-d97d-4e8e-8daa-b3dc1f1c5e4c None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:23.908277 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b5b4e8-d97d-4e8e-8daa-b3dc1f1c5e4c None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:23.908559 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b5b4e8-d97d-4e8e-8daa-b3dc1f1c5e4c None admin] PATH_INFO: `/v3/projects` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:23.909403 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b5b4e8-d97d-4e8e-8daa-b3dc1f1c5e4c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 11:58:23.909723 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b5b4e8-d97d-4e8e-8daa-b3dc1f1c5e4c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:23.911231 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b5b4e8-d97d-4e8e-8daa-b3dc1f1c5e4c None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:23.925239 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 351/695] 10.222.0.23 () {56 vars in 1148 bytes} [Thu May 19 11:58:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:58:23.934066 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b5b4e8-d97d-4e8e-8daa-b3dc1f1c5e4c None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:23.988263 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:58:24.015260 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211b0fee-30e1-4155-9aab-82751cc62d5e None None] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:24.032169 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:58:24.053640 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b5b4e8-d97d-4e8e-8daa-b3dc1f1c5e4c None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:24.054682 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e0c277-7049-4a5c-ad06-e061b6d839c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:24.055862 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e0c277-7049-4a5c-ad06-e061b6d839c9 None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:24.056265 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e0c277-7049-4a5c-ad06-e061b6d839c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:24.056559 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e0c277-7049-4a5c-ad06-e061b6d839c9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:24.069035 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e0c277-7049-4a5c-ad06-e061b6d839c9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:24.070772 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e0c277-7049-4a5c-ad06-e061b6d839c9 None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:24.091104 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663b3553-8f1a-4de9-b4d1-2b567d55159f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:24.091290 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application [None req-c1e0c277-7049-4a5c-ad06-e061b6d839c9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 11:58:24.091290 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 11:58:24.091290 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 11:58:24.091290 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 11:58:24.091290 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 11:58:24.091290 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 11:58:24.091290 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 11:58:24.091290 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 11:58:24.091290 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 11:58:24.091290 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 11:58:24.091290 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 11:58:24.091290 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 11:58:24.091290 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 11:58:24.091290 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 11:58:24.091290 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 11:58:24.092172 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 11:58:24.092172 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 11:58:24.092172 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 11:58:24.092172 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 11:58:24.092172 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 11:58:24.092172 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 11:58:24.092172 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.get_or_create( May 19 11:58:24.092172 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 11:58:24.092172 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application with Lock( May 19 11:58:24.092172 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 11:58:24.092172 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._enter() May 19 11:58:24.092172 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 11:58:24.092172 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 11:58:24.092172 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 11:58:24.092172 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.creator() May 19 11:58:24.092172 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 11:58:24.092172 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application created_value = creator( May 19 11:58:24.093492 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 11:58:24.093492 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 11:58:24.093492 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 11:58:24.093492 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application May 19 11:58:24.093900 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663b3553-8f1a-4de9-b4d1-2b567d55159f None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:24.093900 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663b3553-8f1a-4de9-b4d1-2b567d55159f None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:24.093900 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663b3553-8f1a-4de9-b4d1-2b567d55159f None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:24.094082 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 345/696] 10.222.0.23 () {56 vars in 1108 bytes} [Thu May 19 11:58:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 11:58:24.094157 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663b3553-8f1a-4de9-b4d1-2b567d55159f None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:24.094226 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e0c277-7049-4a5c-ad06-e061b6d839c9 None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:24.095257 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663b3553-8f1a-4de9-b4d1-2b567d55159f None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:24.109797 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application [None req-663b3553-8f1a-4de9-b4d1-2b567d55159f None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. May 19 11:58:24.109797 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 11:58:24.109797 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 11:58:24.109797 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 11:58:24.109797 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 11:58:24.109797 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 11:58:24.109797 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 11:58:24.109797 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 11:58:24.109797 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 19 11:58:24.109797 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application May 19 11:58:24.109797 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 11:58:24.109797 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application May 19 11:58:24.109797 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 11:58:24.109797 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 11:58:24.109797 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 11:58:24.109797 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 11:58:24.110794 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 11:58:24.110794 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 11:58:24.110794 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 11:58:24.110794 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 11:58:24.110794 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 11:58:24.110794 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 11:58:24.110794 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 11:58:24.110794 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 11:58:24.110794 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 11:58:24.110794 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 11:58:24.110794 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 11:58:24.110794 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 11:58:24.110794 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 11:58:24.110794 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 11:58:24.110794 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 11:58:24.110794 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 11:58:24.112625 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.get_or_create( May 19 11:58:24.112625 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 11:58:24.112625 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application with Lock( May 19 11:58:24.112625 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 11:58:24.112625 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._enter() May 19 11:58:24.112625 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 11:58:24.112625 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 11:58:24.112625 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 11:58:24.112625 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.creator() May 19 11:58:24.112625 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 11:58:24.112625 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application created_value = creator( May 19 11:58:24.112625 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 11:58:24.112625 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 11:58:24.112625 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 19 11:58:24.112625 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application May 19 11:58:24.112625 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 352/697] 10.222.0.23 () {56 vars in 1129 bytes} [Thu May 19 11:58:23 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 11:58:24.114697 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663b3553-8f1a-4de9-b4d1-2b567d55159f None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:24.213951 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e0c277-7049-4a5c-ad06-e061b6d839c9 None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:24.221705 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663b3553-8f1a-4de9-b4d1-2b567d55159f None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:24.224132 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:58:24.254104 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b52e4ce-c612-4d71-af17-5723efc50048 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:24.255500 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b52e4ce-c612-4d71-af17-5723efc50048 None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:24.255988 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b52e4ce-c612-4d71-af17-5723efc50048 None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:24.256687 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b52e4ce-c612-4d71-af17-5723efc50048 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:24.257732 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b52e4ce-c612-4d71-af17-5723efc50048 None admin] RBAC: Adding query filter params (role.id=7c59061ba7064a0db6032433fb0eec6b, scope.project.id=d70d25bf69314820b8aaecb65576978a, user.id=15cbfc73ae1243d2a8b1e70f9cd99876) {{(pid=80539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 11:58:24.258201 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b52e4ce-c612-4d71-af17-5723efc50048 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:24.258804 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fddb64c-5543-4428-99c8-1ca9a174683e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:24.259471 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fddb64c-5543-4428-99c8-1ca9a174683e None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:24.259763 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fddb64c-5543-4428-99c8-1ca9a174683e None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:24.260118 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fddb64c-5543-4428-99c8-1ca9a174683e None admin] PATH_INFO: `/v3/domains` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:24.260672 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b52e4ce-c612-4d71-af17-5723efc50048 None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:24.260843 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fddb64c-5543-4428-99c8-1ca9a174683e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 11:58:24.261157 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fddb64c-5543-4428-99c8-1ca9a174683e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:24.262529 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fddb64c-5543-4428-99c8-1ca9a174683e None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:24.269008 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 346/698] 10.222.0.23 () {56 vars in 1374 bytes} [Thu May 19 11:58:24 2022] GET /identity/v3/role_assignments?user.id=15cbfc73ae1243d2a8b1e70f9cd99876&scope.project.id=d70d25bf69314820b8aaecb65576978a&role.id=7c59061ba7064a0db6032433fb0eec6b => generated 608 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:58:24.270133 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b52e4ce-c612-4d71-af17-5723efc50048 None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:24.282882 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 353/699] 10.222.0.23 () {56 vars in 1109 bytes} [Thu May 19 11:58:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:58:24.284430 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:58:24.288378 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fddb64c-5543-4428-99c8-1ca9a174683e None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:24.393115 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b52e4ce-c612-4d71-af17-5723efc50048 None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:24.440425 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fddb64c-5543-4428-99c8-1ca9a174683e None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:24.480238 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ec19ba-14fa-4a7d-92ab-3676a9a7297d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:24.481355 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ec19ba-14fa-4a7d-92ab-3676a9a7297d None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:24.481645 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ec19ba-14fa-4a7d-92ab-3676a9a7297d None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:24.482010 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ec19ba-14fa-4a7d-92ab-3676a9a7297d None admin] PATH_INFO: `/v3/users/nova` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:24.485423 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54b9332-904c-42dd-aec1-e6e00b0802c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:24.486588 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54b9332-904c-42dd-aec1-e6e00b0802c7 None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:24.487080 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54b9332-904c-42dd-aec1-e6e00b0802c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:24.487561 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54b9332-904c-42dd-aec1-e6e00b0802c7 None admin] PATH_INFO: `/v3/services` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:24.488526 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54b9332-904c-42dd-aec1-e6e00b0802c7 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 11:58:24.489063 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54b9332-904c-42dd-aec1-e6e00b0802c7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:24.490712 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54b9332-904c-42dd-aec1-e6e00b0802c7 None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:24.492010 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ec19ba-14fa-4a7d-92ab-3676a9a7297d None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:24.499827 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ec19ba-14fa-4a7d-92ab-3676a9a7297d None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:24.504953 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 347/700] 10.222.0.23 () {56 vars in 1124 bytes} [Thu May 19 11:58:24 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:58:24.510540 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application [None req-56ec19ba-14fa-4a7d-92ab-3676a9a7297d None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 19 11:58:24.510540 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 11:58:24.510540 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 11:58:24.510540 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 11:58:24.510540 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 11:58:24.510540 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 11:58:24.510540 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 11:58:24.510540 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 11:58:24.510540 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 11:58:24.510540 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 11:58:24.510540 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 11:58:24.510540 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 11:58:24.510540 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 11:58:24.510540 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 11:58:24.510540 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 11:58:24.510540 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 11:58:24.511579 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 11:58:24.511579 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 11:58:24.511579 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 11:58:24.511579 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 11:58:24.511579 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 11:58:24.511579 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 11:58:24.511579 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 11:58:24.511579 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 11:58:24.511579 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 11:58:24.511579 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.get_or_create( May 19 11:58:24.511579 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 11:58:24.511579 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application with Lock( May 19 11:58:24.511579 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 11:58:24.511579 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._enter() May 19 11:58:24.511579 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 11:58:24.511579 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 11:58:24.511579 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 11:58:24.511579 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.creator() May 19 11:58:24.512627 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 11:58:24.512627 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application created_value = creator( May 19 11:58:24.512627 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 11:58:24.512627 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 11:58:24.512627 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 11:58:24.512627 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 11:58:24.512627 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 11:58:24.512627 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 11:58:24.512627 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 19 11:58:24.512627 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application May 19 11:58:24.513156 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54b9332-904c-42dd-aec1-e6e00b0802c7 None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:24.513244 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 354/701] 10.222.0.23 () {56 vars in 1093 bytes} [Thu May 19 11:58:24 2022] GET /identity/v3/users/nova => generated 82 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 11:58:24.518534 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ec19ba-14fa-4a7d-92ab-3676a9a7297d None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:24.556936 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:58:24.639367 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ec19ba-14fa-4a7d-92ab-3676a9a7297d None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:24.664108 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:58:24.684585 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81de01e4-19a7-477b-8f92-fd9ead80acb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:24.686142 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81de01e4-19a7-477b-8f92-fd9ead80acb4 None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:24.686582 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81de01e4-19a7-477b-8f92-fd9ead80acb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:24.687012 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81de01e4-19a7-477b-8f92-fd9ead80acb4 None admin] PATH_INFO: `/v3/users` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:24.688265 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81de01e4-19a7-477b-8f92-fd9ead80acb4 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 11:58:24.688760 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81de01e4-19a7-477b-8f92-fd9ead80acb4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:24.690599 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81de01e4-19a7-477b-8f92-fd9ead80acb4 None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:24.699116 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54b9332-904c-42dd-aec1-e6e00b0802c7 None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:24.717562 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 355/702] 10.222.0.23 () {56 vars in 1133 bytes} [Thu May 19 11:58:24 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 11:58:24.722613 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81de01e4-19a7-477b-8f92-fd9ead80acb4 None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:24.743949 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3d7a94-2a1e-40ad-9048-917747227dbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:24.744729 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3d7a94-2a1e-40ad-9048-917747227dbd None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:24.745083 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3d7a94-2a1e-40ad-9048-917747227dbd None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:24.745431 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3d7a94-2a1e-40ad-9048-917747227dbd None admin] PATH_INFO: `/v3/services` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:24.746462 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3d7a94-2a1e-40ad-9048-917747227dbd None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 11:58:24.746833 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3d7a94-2a1e-40ad-9048-917747227dbd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:24.749574 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3d7a94-2a1e-40ad-9048-917747227dbd None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:24.757022 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 348/703] 10.222.0.23 () {56 vars in 1124 bytes} [Thu May 19 11:58:24 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 11:58:24.836979 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:58:24.859421 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81de01e4-19a7-477b-8f92-fd9ead80acb4 None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:24.900054 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541e434f-7195-474f-9e5e-d7b2a649cff0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:24.900793 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541e434f-7195-474f-9e5e-d7b2a649cff0 None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:24.901080 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541e434f-7195-474f-9e5e-d7b2a649cff0 None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:24.901357 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541e434f-7195-474f-9e5e-d7b2a649cff0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:24.986557 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541e434f-7195-474f-9e5e-d7b2a649cff0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:24.988518 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541e434f-7195-474f-9e5e-d7b2a649cff0 None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:25.084180 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:58:25.096959 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 356/704] 10.222.0.23 () {58 vars in 1282 bytes} [Thu May 19 11:58:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 19 11:58:25.102452 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3d7a94-2a1e-40ad-9048-917747227dbd None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:25.220709 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3d7a94-2a1e-40ad-9048-917747227dbd None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:25.256664 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14c4a9f-6409-4b8e-b38e-dabf1cc83006 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:25.257469 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14c4a9f-6409-4b8e-b38e-dabf1cc83006 None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:25.257760 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14c4a9f-6409-4b8e-b38e-dabf1cc83006 None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:25.258038 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14c4a9f-6409-4b8e-b38e-dabf1cc83006 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:25.266052 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:58:25.270627 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14c4a9f-6409-4b8e-b38e-dabf1cc83006 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:25.272138 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14c4a9f-6409-4b8e-b38e-dabf1cc83006 None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:25.290127 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application [None req-d14c4a9f-6409-4b8e-b38e-dabf1cc83006 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 11:58:25.290127 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 11:58:25.290127 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 11:58:25.290127 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 11:58:25.290127 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 11:58:25.290127 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 11:58:25.290127 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 11:58:25.290127 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 11:58:25.290127 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 11:58:25.290127 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 11:58:25.290127 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 11:58:25.290127 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 11:58:25.290127 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 11:58:25.290127 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 11:58:25.290127 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 11:58:25.291086 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 11:58:25.291086 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 11:58:25.291086 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 11:58:25.291086 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 11:58:25.291086 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 11:58:25.291086 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 11:58:25.291086 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.get_or_create( May 19 11:58:25.291086 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 11:58:25.291086 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application with Lock( May 19 11:58:25.291086 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 11:58:25.291086 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._enter() May 19 11:58:25.291086 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 11:58:25.291086 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 11:58:25.291086 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 11:58:25.291086 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.creator() May 19 11:58:25.291086 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 11:58:25.291086 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application created_value = creator( May 19 11:58:25.292369 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 11:58:25.292369 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 11:58:25.292369 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 11:58:25.292369 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application May 19 11:58:25.292369 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 349/705] 10.222.0.23 () {56 vars in 1108 bytes} [Thu May 19 11:58:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 11:58:25.297418 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541e434f-7195-474f-9e5e-d7b2a649cff0 None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:25.486704 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541e434f-7195-474f-9e5e-d7b2a649cff0 None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:25.535251 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5dee71e-30ae-4c35-8dcb-e7f110177c3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:25.535953 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5dee71e-30ae-4c35-8dcb-e7f110177c3f None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:25.536241 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5dee71e-30ae-4c35-8dcb-e7f110177c3f None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:25.536519 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5dee71e-30ae-4c35-8dcb-e7f110177c3f None admin] PATH_INFO: `/v3/domains` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:25.537252 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5dee71e-30ae-4c35-8dcb-e7f110177c3f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 11:58:25.537557 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5dee71e-30ae-4c35-8dcb-e7f110177c3f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:25.538938 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5dee71e-30ae-4c35-8dcb-e7f110177c3f None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:25.555177 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 357/706] 10.222.0.23 () {56 vars in 1109 bytes} [Thu May 19 11:58:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:58:25.560923 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14c4a9f-6409-4b8e-b38e-dabf1cc83006 None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:25.635048 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf30b50-c006-4856-b9f7-f0517dde2ee4 None None] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:25.635696 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf30b50-c006-4856-b9f7-f0517dde2ee4 None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:25.636304 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf30b50-c006-4856-b9f7-f0517dde2ee4 None None] PATH_INFO: `/` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:25.638157 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 358/707] 10.222.0.23 () {54 vars in 926 bytes} [Thu May 19 11:58:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 11:58:25.643850 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8694e7-652e-4240-a23c-9a517e30237f None None] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:25.644860 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8694e7-652e-4240-a23c-9a517e30237f None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:25.645348 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8694e7-652e-4240-a23c-9a517e30237f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:25.684994 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.auth.core [None req-ad8694e7-652e-4240-a23c-9a517e30237f None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 11:58:25.687401 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-ad8694e7-652e-4240-a23c-9a517e30237f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 11:58:25.688693 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14c4a9f-6409-4b8e-b38e-dabf1cc83006 None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:25.727929 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bea3003-c81a-4d4d-a440-75500dd2f298 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:25.728734 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bea3003-c81a-4d4d-a440-75500dd2f298 None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:25.729021 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bea3003-c81a-4d4d-a440-75500dd2f298 None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:25.729300 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bea3003-c81a-4d4d-a440-75500dd2f298 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:25.737196 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bea3003-c81a-4d4d-a440-75500dd2f298 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:25.740592 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bea3003-c81a-4d4d-a440-75500dd2f298 None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:25.754365 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application [None req-2bea3003-c81a-4d4d-a440-75500dd2f298 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 11:58:25.754365 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 11:58:25.754365 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 11:58:25.754365 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 11:58:25.754365 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 11:58:25.754365 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 11:58:25.754365 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 11:58:25.754365 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 11:58:25.754365 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 11:58:25.754365 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 11:58:25.754365 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 11:58:25.754365 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 11:58:25.754365 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 11:58:25.754365 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 11:58:25.754365 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 11:58:25.755845 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 11:58:25.755845 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 11:58:25.755845 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 11:58:25.755845 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 11:58:25.755845 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 11:58:25.755845 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 11:58:25.755845 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.get_or_create( May 19 11:58:25.755845 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 11:58:25.755845 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application with Lock( May 19 11:58:25.755845 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 11:58:25.755845 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._enter() May 19 11:58:25.755845 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 11:58:25.755845 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 11:58:25.755845 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 11:58:25.755845 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.creator() May 19 11:58:25.755845 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 11:58:25.755845 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application created_value = creator( May 19 11:58:25.761106 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 11:58:25.761106 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 11:58:25.761106 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 11:58:25.761106 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 11:58:25.761106 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 11:58:25.761106 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 11:58:25.761106 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 11:58:25.761106 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application May 19 11:58:25.764827 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 350/708] 10.222.0.23 () {56 vars in 1111 bytes} [Thu May 19 11:58:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 11:58:25.765860 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bea3003-c81a-4d4d-a440-75500dd2f298 None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:25.769191 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:58:25.835297 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 359/709] 10.222.0.23 () {58 vars in 1023 bytes} [Thu May 19 11:58:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 11:58:25.868188 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:58:25.873802 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8694e7-652e-4240-a23c-9a517e30237f None None] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:25.914069 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bea3003-c81a-4d4d-a440-75500dd2f298 None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:25.963266 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54dde56-a28c-4daf-946b-3f493ed06614 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:25.964075 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54dde56-a28c-4daf-946b-3f493ed06614 None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:25.964459 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54dde56-a28c-4daf-946b-3f493ed06614 None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:25.964835 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54dde56-a28c-4daf-946b-3f493ed06614 None admin] PATH_INFO: `/v3/projects` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:25.965824 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54dde56-a28c-4daf-946b-3f493ed06614 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 11:58:25.966296 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54dde56-a28c-4daf-946b-3f493ed06614 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:25.968791 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54dde56-a28c-4daf-946b-3f493ed06614 None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:25.982169 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 351/710] 10.222.0.23 () {56 vars in 1148 bytes} [Thu May 19 11:58:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:58:25.983523 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c80ddc-357c-4e50-aa33-c6120c97c0b1 None None] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:25.983961 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c80ddc-357c-4e50-aa33-c6120c97c0b1 None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:25.984347 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c80ddc-357c-4e50-aa33-c6120c97c0b1 None None] PATH_INFO: `/` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:25.986143 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 352/711] 10.222.0.23 () {54 vars in 926 bytes} [Thu May 19 11:58:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 11:58:25.991214 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c80ddc-357c-4e50-aa33-c6120c97c0b1 None None] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:26.034418 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8694e7-652e-4240-a23c-9a517e30237f None None] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:26.078318 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc024fee-c29e-4a04-ab46-5949fa170e10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:26.079719 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc024fee-c29e-4a04-ab46-5949fa170e10 None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:26.080452 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc024fee-c29e-4a04-ab46-5949fa170e10 None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:26.080879 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc024fee-c29e-4a04-ab46-5949fa170e10 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:26.096577 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc024fee-c29e-4a04-ab46-5949fa170e10 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:26.098708 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc024fee-c29e-4a04-ab46-5949fa170e10 None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:26.106652 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c80ddc-357c-4e50-aa33-c6120c97c0b1 None None] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:26.119357 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:58:26.124010 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application [None req-cc024fee-c29e-4a04-ab46-5949fa170e10 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 11:58:26.124010 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 11:58:26.124010 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 11:58:26.124010 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 11:58:26.124010 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 11:58:26.124010 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 11:58:26.124010 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 11:58:26.124010 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 11:58:26.124010 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 11:58:26.124010 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 11:58:26.124010 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 11:58:26.124010 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 11:58:26.124010 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 11:58:26.124010 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 11:58:26.124010 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 11:58:26.124914 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 11:58:26.124914 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 11:58:26.124914 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 11:58:26.124914 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 11:58:26.124914 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 11:58:26.124914 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 11:58:26.124914 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.get_or_create( May 19 11:58:26.124914 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 11:58:26.124914 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application with Lock( May 19 11:58:26.124914 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 11:58:26.124914 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._enter() May 19 11:58:26.124914 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 11:58:26.124914 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 11:58:26.124914 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 11:58:26.124914 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.creator() May 19 11:58:26.124914 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 11:58:26.124914 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application created_value = creator( May 19 11:58:26.125862 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 11:58:26.125862 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 11:58:26.125862 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 11:58:26.125862 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application May 19 11:58:26.127613 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 360/712] 10.222.0.23 () {56 vars in 1108 bytes} [Thu May 19 11:58:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 11:58:26.131328 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e54a10-0b25-42c8-89a8-52debe28cd85 None None] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:26.131328 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e54a10-0b25-42c8-89a8-52debe28cd85 None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:26.131328 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e54a10-0b25-42c8-89a8-52debe28cd85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:26.152163 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.auth.core [None req-c2e54a10-0b25-42c8-89a8-52debe28cd85 None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 11:58:26.153160 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9d91a4-948d-42e2-b1e2-cb3ec3a30ac0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:26.153854 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-c2e54a10-0b25-42c8-89a8-52debe28cd85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 11:58:26.153982 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9d91a4-948d-42e2-b1e2-cb3ec3a30ac0 None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:26.154965 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9d91a4-948d-42e2-b1e2-cb3ec3a30ac0 None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:26.155373 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9d91a4-948d-42e2-b1e2-cb3ec3a30ac0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:26.156499 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d9d91a4-948d-42e2-b1e2-cb3ec3a30ac0 None admin] RBAC: Adding query filter params (role.id=9db5671c65d3414ea2682afa2ccf1403, scope.project.id=d70d25bf69314820b8aaecb65576978a, user.id=34a98d75d84b4c67981377e0f26eca62) {{(pid=80539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 11:58:26.156916 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d9d91a4-948d-42e2-b1e2-cb3ec3a30ac0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:26.158903 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d9d91a4-948d-42e2-b1e2-cb3ec3a30ac0 None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:26.165661 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 353/713] 10.222.0.23 () {56 vars in 1374 bytes} [Thu May 19 11:58:25 2022] GET /identity/v3/role_assignments?user.id=34a98d75d84b4c67981377e0f26eca62&scope.project.id=d70d25bf69314820b8aaecb65576978a&role.id=9db5671c65d3414ea2682afa2ccf1403 => generated 259 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:58:26.166492 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9d91a4-948d-42e2-b1e2-cb3ec3a30ac0 None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:26.196163 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:58:26.291116 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 361/714] 10.222.0.23 () {58 vars in 1023 bytes} [Thu May 19 11:58:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 11:58:26.325636 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e54a10-0b25-42c8-89a8-52debe28cd85 None None] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:26.330576 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9d91a4-948d-42e2-b1e2-cb3ec3a30ac0 None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:26.383532 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3381bbe-c6fd-4e26-99a7-d7512fdea3a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:26.384730 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3381bbe-c6fd-4e26-99a7-d7512fdea3a5 None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:26.385427 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3381bbe-c6fd-4e26-99a7-d7512fdea3a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:26.387334 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3381bbe-c6fd-4e26-99a7-d7512fdea3a5 None admin] PATH_INFO: `/v3/domains` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:26.389362 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3381bbe-c6fd-4e26-99a7-d7512fdea3a5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 11:58:26.389975 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3381bbe-c6fd-4e26-99a7-d7512fdea3a5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:26.392660 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3381bbe-c6fd-4e26-99a7-d7512fdea3a5 None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:26.417342 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 354/715] 10.222.0.23 () {56 vars in 1109 bytes} [Thu May 19 11:58:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:58:26.424239 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3381bbe-c6fd-4e26-99a7-d7512fdea3a5 None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:26.463360 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:58:26.527510 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e54a10-0b25-42c8-89a8-52debe28cd85 None None] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:26.567160 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763bd2ed-f95e-43f5-ac74-022bacd8e803 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:26.567992 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763bd2ed-f95e-43f5-ac74-022bacd8e803 None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:26.568277 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763bd2ed-f95e-43f5-ac74-022bacd8e803 None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:26.568554 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763bd2ed-f95e-43f5-ac74-022bacd8e803 None admin] PATH_INFO: `/v3/services/network` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:26.569349 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763bd2ed-f95e-43f5-ac74-022bacd8e803 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:26.571052 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763bd2ed-f95e-43f5-ac74-022bacd8e803 None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:26.582432 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3381bbe-c6fd-4e26-99a7-d7512fdea3a5 None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:26.583171 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application [None req-763bd2ed-f95e-43f5-ac74-022bacd8e803 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. May 19 11:58:26.583171 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 11:58:26.583171 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 11:58:26.583171 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 11:58:26.583171 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 11:58:26.583171 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 11:58:26.583171 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 11:58:26.583171 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 11:58:26.583171 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 19 11:58:26.583171 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application May 19 11:58:26.583171 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 11:58:26.583171 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application May 19 11:58:26.583171 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 11:58:26.583171 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 11:58:26.583171 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 11:58:26.583171 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 11:58:26.583171 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 11:58:26.584183 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 11:58:26.584183 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 11:58:26.584183 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 11:58:26.584183 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 11:58:26.584183 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 11:58:26.584183 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 11:58:26.584183 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 11:58:26.584183 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 11:58:26.584183 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 11:58:26.584183 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 11:58:26.584183 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 11:58:26.584183 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 11:58:26.584183 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 11:58:26.584183 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 11:58:26.584183 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 11:58:26.584183 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.get_or_create( May 19 11:58:26.585036 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 11:58:26.585036 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application with Lock( May 19 11:58:26.585036 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 11:58:26.585036 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._enter() May 19 11:58:26.585036 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 11:58:26.585036 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 11:58:26.585036 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 11:58:26.585036 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.creator() May 19 11:58:26.585036 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 11:58:26.585036 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application created_value = creator( May 19 11:58:26.585036 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 11:58:26.585036 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 11:58:26.585036 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 19 11:58:26.585036 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application May 19 11:58:26.585036 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 362/716] 10.222.0.23 () {56 vars in 1111 bytes} [Thu May 19 11:58:26 2022] GET /identity/v3/services/network => generated 88 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 11:58:26.597013 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763bd2ed-f95e-43f5-ac74-022bacd8e803 None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:26.623742 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:58:26.640331 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa73a48-a470-41f1-bf16-ff757b641865 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:26.641175 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa73a48-a470-41f1-bf16-ff757b641865 None admin] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:26.641583 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa73a48-a470-41f1-bf16-ff757b641865 None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:26.641960 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa73a48-a470-41f1-bf16-ff757b641865 None admin] PATH_INFO: `/v3/users` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:26.642904 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa73a48-a470-41f1-bf16-ff757b641865 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:26.645257 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa73a48-a470-41f1-bf16-ff757b641865 None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:26.718326 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763bd2ed-f95e-43f5-ac74-022bacd8e803 None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:26.746792 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 355/717] 10.222.0.23 () {60 vars in 1132 bytes} [Thu May 19 11:58:26 2022] POST /identity/v3/users => generated 254 bytes in 323 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 11:58:26.758833 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce20843e-da3c-456c-90cc-1f59f6bbf9de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:26.759926 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce20843e-da3c-456c-90cc-1f59f6bbf9de None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:26.760386 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce20843e-da3c-456c-90cc-1f59f6bbf9de None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:26.760825 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce20843e-da3c-456c-90cc-1f59f6bbf9de None admin] PATH_INFO: `/v3/services` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:26.761801 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce20843e-da3c-456c-90cc-1f59f6bbf9de None admin] RBAC: Adding query filter params (name=network) {{(pid=80540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 11:58:26.762249 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce20843e-da3c-456c-90cc-1f59f6bbf9de None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:26.764397 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce20843e-da3c-456c-90cc-1f59f6bbf9de None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:26.770306 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 363/718] 10.222.0.23 () {56 vars in 1112 bytes} [Thu May 19 11:58:26 2022] GET /identity/v3/services?name=network => generated 124 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:58:26.775918 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa73a48-a470-41f1-bf16-ff757b641865 None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:26.939086 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:58:26.959258 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa73a48-a470-41f1-bf16-ff757b641865 None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:27.002447 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1993788f-1fc9-4f59-b0b2-f55204b725aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:27.003601 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1993788f-1fc9-4f59-b0b2-f55204b725aa None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:27.004582 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1993788f-1fc9-4f59-b0b2-f55204b725aa None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:27.005058 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1993788f-1fc9-4f59-b0b2-f55204b725aa None admin] PATH_INFO: `/v3/services` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:27.005970 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1993788f-1fc9-4f59-b0b2-f55204b725aa None admin] RBAC: Adding query filter params (type=network) {{(pid=80539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 11:58:27.006488 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1993788f-1fc9-4f59-b0b2-f55204b725aa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:27.008128 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1993788f-1fc9-4f59-b0b2-f55204b725aa None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:27.017277 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 356/719] 10.222.0.23 () {56 vars in 1112 bytes} [Thu May 19 11:58:26 2022] GET /identity/v3/services?type=network => generated 124 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:58:27.386415 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46b0a72-2469-429c-98fa-e48fdda59081 None None] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:27.386748 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46b0a72-2469-429c-98fa-e48fdda59081 None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:27.386983 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46b0a72-2469-429c-98fa-e48fdda59081 None None] PATH_INFO: `/` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:27.388679 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 364/720] 10.222.0.23 () {54 vars in 926 bytes} [Thu May 19 11:58:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 11:58:27.401776 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f30a6c-2842-4b6c-9049-c1f64d91f5ba None None] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:27.402070 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f30a6c-2842-4b6c-9049-c1f64d91f5ba None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:27.402338 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f30a6c-2842-4b6c-9049-c1f64d91f5ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:27.428116 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.auth.core [None req-c3f30a6c-2842-4b6c-9049-c1f64d91f5ba None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 11:58:27.429299 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-c3f30a6c-2842-4b6c-9049-c1f64d91f5ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 11:58:27.533510 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 357/721] 10.222.0.23 () {58 vars in 1023 bytes} [Thu May 19 11:58:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 11:58:27.564909 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46b0a72-2469-429c-98fa-e48fdda59081 None None] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:27.682575 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:58:27.708482 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46b0a72-2469-429c-98fa-e48fdda59081 None None] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:27.744241 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfe3898-49c9-4ad2-b346-49e059306fab None None] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:27.744881 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfe3898-49c9-4ad2-b346-49e059306fab None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:27.745396 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfe3898-49c9-4ad2-b346-49e059306fab None None] PATH_INFO: `/` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:27.747271 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 358/722] 10.222.0.23 () {54 vars in 926 bytes} [Thu May 19 11:58:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 11:58:27.752759 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaef5372-8e6f-41e5-9c12-38ed74a31e58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:27.753565 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaef5372-8e6f-41e5-9c12-38ed74a31e58 None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:27.753851 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaef5372-8e6f-41e5-9c12-38ed74a31e58 None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:27.754135 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaef5372-8e6f-41e5-9c12-38ed74a31e58 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:27.754919 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2301e62-1e8f-4234-b301-58a2459d8f73 None None] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:27.755431 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2301e62-1e8f-4234-b301-58a2459d8f73 None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:27.755986 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2301e62-1e8f-4234-b301-58a2459d8f73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:27.781031 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaef5372-8e6f-41e5-9c12-38ed74a31e58 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 11:58:27.792158 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application [None req-eaef5372-8e6f-41e5-9c12-38ed74a31e58 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 11:58:27.792158 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 11:58:27.792158 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 11:58:27.792158 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 11:58:27.792158 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 11:58:27.792158 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 11:58:27.792158 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 11:58:27.792158 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 11:58:27.792158 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 11:58:27.792158 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 11:58:27.792158 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 11:58:27.792158 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 11:58:27.792158 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 11:58:27.792158 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 11:58:27.792158 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 11:58:27.792158 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 11:58:27.793149 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 11:58:27.793149 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 11:58:27.793149 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 11:58:27.793149 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 11:58:27.793149 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 11:58:27.793149 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 11:58:27.793149 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 11:58:27.793149 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 11:58:27.793149 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 11:58:27.793149 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.get_or_create( May 19 11:58:27.793149 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 11:58:27.793149 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application with Lock( May 19 11:58:27.793149 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 11:58:27.793149 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._enter() May 19 11:58:27.793149 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 11:58:27.793149 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 11:58:27.793980 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 11:58:27.793980 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.creator() May 19 11:58:27.793980 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 11:58:27.793980 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application created_value = creator( May 19 11:58:27.793980 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 11:58:27.793980 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 11:58:27.793980 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 11:58:27.793980 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 11:58:27.793980 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 11:58:27.793980 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 11:58:27.793980 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 11:58:27.793980 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application May 19 11:58:27.805757 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 365/723] 10.222.0.23 () {56 vars in 1096 bytes} [Thu May 19 11:58:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 11:58:27.811910 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaef5372-8e6f-41e5-9c12-38ed74a31e58 None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:27.915367 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.auth.core [None req-c2301e62-1e8f-4234-b301-58a2459d8f73 None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 11:58:27.917384 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-c2301e62-1e8f-4234-b301-58a2459d8f73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 11:58:28.017793 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 359/724] 10.222.0.23 () {58 vars in 1023 bytes} [Thu May 19 11:58:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 265 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 11:58:28.040041 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaef5372-8e6f-41e5-9c12-38ed74a31e58 None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:28.056962 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2301e62-1e8f-4234-b301-58a2459d8f73 None None] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:28.079819 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:58:28.110977 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1503e71e-3a88-47d2-8119-15a089bd6913 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:28.111659 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1503e71e-3a88-47d2-8119-15a089bd6913 None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:28.111979 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1503e71e-3a88-47d2-8119-15a089bd6913 None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:28.112259 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1503e71e-3a88-47d2-8119-15a089bd6913 None admin] PATH_INFO: `/v3/roles` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:28.113015 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1503e71e-3a88-47d2-8119-15a089bd6913 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 11:58:28.113325 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1503e71e-3a88-47d2-8119-15a089bd6913 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:28.116236 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1503e71e-3a88-47d2-8119-15a089bd6913 None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:28.133567 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 366/725] 10.222.0.23 () {56 vars in 1099 bytes} [Thu May 19 11:58:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 323 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 11:58:28.139944 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1503e71e-3a88-47d2-8119-15a089bd6913 None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:28.148563 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:58:28.237244 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2301e62-1e8f-4234-b301-58a2459d8f73 None None] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:28.311069 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e96c4a2-5ef6-499d-9c9d-a7a78b615585 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:28.311746 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e96c4a2-5ef6-499d-9c9d-a7a78b615585 None admin] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:28.312055 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e96c4a2-5ef6-499d-9c9d-a7a78b615585 None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:28.312329 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e96c4a2-5ef6-499d-9c9d-a7a78b615585 None admin] PATH_INFO: `/v3/services` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:28.313064 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e96c4a2-5ef6-499d-9c9d-a7a78b615585 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:28.315396 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e96c4a2-5ef6-499d-9c9d-a7a78b615585 None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:28.322747 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1503e71e-3a88-47d2-8119-15a089bd6913 None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:28.365884 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692d2f3a-6431-409d-b709-2fb8892a355c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:28.366691 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692d2f3a-6431-409d-b709-2fb8892a355c None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:28.366980 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692d2f3a-6431-409d-b709-2fb8892a355c None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:28.367259 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692d2f3a-6431-409d-b709-2fb8892a355c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:28.380162 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692d2f3a-6431-409d-b709-2fb8892a355c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:28.382343 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692d2f3a-6431-409d-b709-2fb8892a355c None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:28.405492 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application [None req-692d2f3a-6431-409d-b709-2fb8892a355c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 11:58:28.405492 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 11:58:28.405492 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 11:58:28.405492 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 11:58:28.405492 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 11:58:28.405492 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 11:58:28.405492 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 11:58:28.405492 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 11:58:28.405492 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 11:58:28.405492 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 11:58:28.405492 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 11:58:28.405492 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 11:58:28.405492 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 11:58:28.405492 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 11:58:28.405492 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 11:58:28.406455 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 11:58:28.406455 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 11:58:28.406455 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 11:58:28.406455 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 11:58:28.406455 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 11:58:28.406455 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 11:58:28.406455 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.get_or_create( May 19 11:58:28.406455 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 11:58:28.406455 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application with Lock( May 19 11:58:28.406455 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 11:58:28.406455 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._enter() May 19 11:58:28.406455 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 11:58:28.406455 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 11:58:28.406455 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 11:58:28.406455 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.creator() May 19 11:58:28.406455 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 11:58:28.406455 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application created_value = creator( May 19 11:58:28.407340 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 11:58:28.407340 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 11:58:28.407340 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 11:58:28.407340 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application May 19 11:58:28.408414 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 367/726] 10.222.0.23 () {56 vars in 1108 bytes} [Thu May 19 11:58:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 11:58:28.413399 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 360/727] 10.222.0.23 () {60 vars in 1141 bytes} [Thu May 19 11:58:28 2022] POST /identity/v3/services => generated 251 bytes in 357 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 11:58:28.414544 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692d2f3a-6431-409d-b709-2fb8892a355c None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:28.480639 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:58:28.533836 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692d2f3a-6431-409d-b709-2fb8892a355c None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:28.585205 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c7ad27-452d-40f6-9e12-14dd9f8df403 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:28.585905 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c7ad27-452d-40f6-9e12-14dd9f8df403 None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:28.586196 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c7ad27-452d-40f6-9e12-14dd9f8df403 None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:28.586481 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c7ad27-452d-40f6-9e12-14dd9f8df403 None admin] PATH_INFO: `/v3/domains` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:28.587237 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c7ad27-452d-40f6-9e12-14dd9f8df403 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 11:58:28.587552 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c7ad27-452d-40f6-9e12-14dd9f8df403 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:28.589043 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c7ad27-452d-40f6-9e12-14dd9f8df403 None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:28.611947 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 368/728] 10.222.0.23 () {56 vars in 1109 bytes} [Thu May 19 11:58:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:58:28.617488 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e96c4a2-5ef6-499d-9c9d-a7a78b615585 None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:28.719854 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:58:28.756443 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e96c4a2-5ef6-499d-9c9d-a7a78b615585 None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:28.798731 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f18565b-a7ea-47a8-9920-2543d469d445 None None] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:28.799220 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f18565b-a7ea-47a8-9920-2543d469d445 None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:28.799220 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f18565b-a7ea-47a8-9920-2543d469d445 None None] PATH_INFO: `/` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:28.800587 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 369/729] 10.222.0.23 () {54 vars in 926 bytes} [Thu May 19 11:58:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 11:58:28.812952 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701181dc-6b2f-4b00-9648-c6c10c79f4d0 None None] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:28.813579 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701181dc-6b2f-4b00-9648-c6c10c79f4d0 None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:28.813687 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685c089e-f7d3-4a1b-b666-0d31f8b2eda4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:28.814225 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701181dc-6b2f-4b00-9648-c6c10c79f4d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:28.814693 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685c089e-f7d3-4a1b-b666-0d31f8b2eda4 None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:28.814983 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685c089e-f7d3-4a1b-b666-0d31f8b2eda4 None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:28.815261 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685c089e-f7d3-4a1b-b666-0d31f8b2eda4 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:28.828065 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685c089e-f7d3-4a1b-b666-0d31f8b2eda4 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:28.829540 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685c089e-f7d3-4a1b-b666-0d31f8b2eda4 None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:28.839958 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.auth.core [None req-701181dc-6b2f-4b00-9648-c6c10c79f4d0 None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 11:58:28.841201 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-701181dc-6b2f-4b00-9648-c6c10c79f4d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 11:58:28.845395 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application [None req-685c089e-f7d3-4a1b-b666-0d31f8b2eda4 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 19 11:58:28.845395 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 11:58:28.845395 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 11:58:28.845395 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 11:58:28.845395 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 11:58:28.845395 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 11:58:28.845395 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 11:58:28.845395 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 11:58:28.845395 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 11:58:28.845395 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 11:58:28.845395 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 11:58:28.845395 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 11:58:28.845395 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 11:58:28.845395 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 11:58:28.845395 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 11:58:28.845395 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 11:58:28.846361 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 11:58:28.846361 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 11:58:28.846361 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 11:58:28.846361 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 11:58:28.846361 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 11:58:28.846361 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 11:58:28.846361 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 11:58:28.846361 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 11:58:28.846361 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 11:58:28.846361 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.get_or_create( May 19 11:58:28.846361 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 11:58:28.846361 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application with Lock( May 19 11:58:28.846361 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 11:58:28.846361 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._enter() May 19 11:58:28.846361 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 11:58:28.846361 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 11:58:28.846361 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 11:58:28.846361 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.creator() May 19 11:58:28.849030 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 11:58:28.849030 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application created_value = creator( May 19 11:58:28.849030 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 11:58:28.849030 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 11:58:28.849030 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 11:58:28.849030 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 11:58:28.849030 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 11:58:28.849030 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 11:58:28.849030 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 19 11:58:28.849030 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application May 19 11:58:28.849030 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 361/730] 10.222.0.23 () {56 vars in 1096 bytes} [Thu May 19 11:58:28 2022] GET /identity/v3/users/swift => generated 83 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 11:58:28.860147 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685c089e-f7d3-4a1b-b666-0d31f8b2eda4 None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:28.938293 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 370/731] 10.222.0.23 () {58 vars in 1023 bytes} [Thu May 19 11:58:28 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 11:58:28.958895 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701181dc-6b2f-4b00-9648-c6c10c79f4d0 None None] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:28.972759 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685c089e-f7d3-4a1b-b666-0d31f8b2eda4 None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:29.009426 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e958141e-363d-4a69-8ecb-104793a62fd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:29.010155 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e958141e-363d-4a69-8ecb-104793a62fd7 None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:29.010445 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e958141e-363d-4a69-8ecb-104793a62fd7 None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:29.010722 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e958141e-363d-4a69-8ecb-104793a62fd7 None admin] PATH_INFO: `/v3/users` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:29.011080 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:58:29.011571 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e958141e-363d-4a69-8ecb-104793a62fd7 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 11:58:29.011913 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e958141e-363d-4a69-8ecb-104793a62fd7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:29.013364 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e958141e-363d-4a69-8ecb-104793a62fd7 None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:29.015588 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:58:29.033450 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 362/732] 10.222.0.23 () {56 vars in 1135 bytes} [Thu May 19 11:58:28 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 11:58:29.039664 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e958141e-363d-4a69-8ecb-104793a62fd7 None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:29.087365 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701181dc-6b2f-4b00-9648-c6c10c79f4d0 None None] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:29.128243 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef8ddbd-3316-4538-b12c-f26b27ddc677 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:29.129255 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef8ddbd-3316-4538-b12c-f26b27ddc677 None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:29.129642 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef8ddbd-3316-4538-b12c-f26b27ddc677 None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:29.130023 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef8ddbd-3316-4538-b12c-f26b27ddc677 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:29.142775 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef8ddbd-3316-4538-b12c-f26b27ddc677 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 11:58:29.151156 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application [None req-2ef8ddbd-3316-4538-b12c-f26b27ddc677 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 11:58:29.151156 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 11:58:29.151156 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 11:58:29.151156 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 11:58:29.151156 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 11:58:29.151156 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 11:58:29.151156 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 11:58:29.151156 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 11:58:29.151156 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 11:58:29.151156 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 11:58:29.151156 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 11:58:29.151156 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 11:58:29.151156 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 11:58:29.151156 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 11:58:29.151156 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 11:58:29.151156 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 11:58:29.152283 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 11:58:29.152283 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 11:58:29.152283 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 11:58:29.152283 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 11:58:29.152283 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 11:58:29.152283 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 11:58:29.152283 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 11:58:29.152283 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 11:58:29.152283 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 11:58:29.152283 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.get_or_create( May 19 11:58:29.152283 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 11:58:29.152283 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application with Lock( May 19 11:58:29.152283 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 11:58:29.152283 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._enter() May 19 11:58:29.152283 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 11:58:29.152283 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 11:58:29.153135 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 11:58:29.153135 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.creator() May 19 11:58:29.153135 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 11:58:29.153135 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application created_value = creator( May 19 11:58:29.153135 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 11:58:29.153135 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 11:58:29.153135 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 11:58:29.153135 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 11:58:29.153135 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 11:58:29.153135 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 11:58:29.153135 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 11:58:29.153135 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application May 19 11:58:29.153768 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 371/733] 10.222.0.23 () {56 vars in 1096 bytes} [Thu May 19 11:58:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 11:58:29.160037 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef8ddbd-3316-4538-b12c-f26b27ddc677 None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:29.180088 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e958141e-363d-4a69-8ecb-104793a62fd7 None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:29.244158 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbacc5c7-3570-4458-b8e3-d995775b85e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:29.245201 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbacc5c7-3570-4458-b8e3-d995775b85e3 None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:29.245714 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbacc5c7-3570-4458-b8e3-d995775b85e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:29.246192 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbacc5c7-3570-4458-b8e3-d995775b85e3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:29.358594 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef8ddbd-3316-4538-b12c-f26b27ddc677 None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:29.367217 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:58:29.396166 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:58:29.403360 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbacc5c7-3570-4458-b8e3-d995775b85e3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:29.405420 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbacc5c7-3570-4458-b8e3-d995775b85e3 None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:29.432518 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1d074d-2083-492a-b7ed-219f589f10fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:29.433212 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1d074d-2083-492a-b7ed-219f589f10fc None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:29.433569 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1d074d-2083-492a-b7ed-219f589f10fc None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:29.433851 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1d074d-2083-492a-b7ed-219f589f10fc None admin] PATH_INFO: `/v3/roles` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:29.434672 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1d074d-2083-492a-b7ed-219f589f10fc None admin] RBAC: Adding query filter params (name=admin) {{(pid=80540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 11:58:29.434987 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1d074d-2083-492a-b7ed-219f589f10fc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:29.436946 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1d074d-2083-492a-b7ed-219f589f10fc None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:29.467982 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 372/734] 10.222.0.23 () {56 vars in 1099 bytes} [Thu May 19 11:58:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:58:29.470158 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1d074d-2083-492a-b7ed-219f589f10fc None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:29.591037 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1d074d-2083-492a-b7ed-219f589f10fc None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:29.628470 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 363/735] 10.222.0.23 () {58 vars in 1282 bytes} [Thu May 19 11:58:29 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 590 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 11:58:29.639399 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbacc5c7-3570-4458-b8e3-d995775b85e3 None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:29.646519 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9e8e5c-a07f-4ebb-a6af-0f7e3f424f2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:29.647343 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9e8e5c-a07f-4ebb-a6af-0f7e3f424f2f None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:29.647628 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9e8e5c-a07f-4ebb-a6af-0f7e3f424f2f None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:29.647954 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9e8e5c-a07f-4ebb-a6af-0f7e3f424f2f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:29.662763 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9e8e5c-a07f-4ebb-a6af-0f7e3f424f2f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:29.664734 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9e8e5c-a07f-4ebb-a6af-0f7e3f424f2f None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:29.688582 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application [None req-3b9e8e5c-a07f-4ebb-a6af-0f7e3f424f2f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 11:58:29.688582 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 11:58:29.688582 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 11:58:29.688582 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 11:58:29.688582 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 11:58:29.688582 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 11:58:29.688582 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 11:58:29.688582 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 11:58:29.688582 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 11:58:29.688582 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 11:58:29.688582 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 11:58:29.688582 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 11:58:29.688582 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 11:58:29.688582 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 11:58:29.688582 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 11:58:29.689625 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 11:58:29.689625 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 11:58:29.689625 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 11:58:29.689625 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 11:58:29.689625 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 11:58:29.689625 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 11:58:29.689625 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.get_or_create( May 19 11:58:29.689625 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 11:58:29.689625 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application with Lock( May 19 11:58:29.689625 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 11:58:29.689625 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._enter() May 19 11:58:29.689625 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 11:58:29.689625 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 11:58:29.689625 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 11:58:29.689625 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.creator() May 19 11:58:29.689625 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 11:58:29.689625 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application created_value = creator( May 19 11:58:29.690570 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 11:58:29.690570 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 11:58:29.690570 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 11:58:29.690570 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application May 19 11:58:29.694517 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 373/736] 10.222.0.23 () {56 vars in 1108 bytes} [Thu May 19 11:58:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 11:58:29.696568 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b7cd77-998b-493d-8d46-11b66c338030 None None] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:29.696860 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b7cd77-998b-493d-8d46-11b66c338030 None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:29.697139 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b7cd77-998b-493d-8d46-11b66c338030 None None] PATH_INFO: `/` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:29.698654 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 374/737] 10.222.0.23 () {54 vars in 926 bytes} [Thu May 19 11:58:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 11:58:29.699191 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b7cd77-998b-493d-8d46-11b66c338030 None None] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:29.726142 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:58:29.760250 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbacc5c7-3570-4458-b8e3-d995775b85e3 None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:29.764449 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:58:29.798597 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469c9582-42ae-4d02-8765-0f5763b6b867 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:29.798813 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469c9582-42ae-4d02-8765-0f5763b6b867 None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:29.799096 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469c9582-42ae-4d02-8765-0f5763b6b867 None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:29.799354 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469c9582-42ae-4d02-8765-0f5763b6b867 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:29.810923 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469c9582-42ae-4d02-8765-0f5763b6b867 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:29.814298 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469c9582-42ae-4d02-8765-0f5763b6b867 None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:29.832479 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application [None req-469c9582-42ae-4d02-8765-0f5763b6b867 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 11:58:29.832479 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 11:58:29.832479 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 11:58:29.832479 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 11:58:29.832479 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 11:58:29.832479 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 11:58:29.832479 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 11:58:29.832479 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 11:58:29.832479 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 11:58:29.832479 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 11:58:29.832479 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 11:58:29.832479 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 11:58:29.832479 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 11:58:29.832479 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 11:58:29.832479 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 11:58:29.833599 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 11:58:29.833599 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 11:58:29.833599 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 11:58:29.833599 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 11:58:29.833599 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 11:58:29.833599 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 11:58:29.833599 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.get_or_create( May 19 11:58:29.833599 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 11:58:29.833599 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application with Lock( May 19 11:58:29.833599 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 11:58:29.833599 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._enter() May 19 11:58:29.833599 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 11:58:29.833599 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 11:58:29.833599 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 11:58:29.833599 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.creator() May 19 11:58:29.833599 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 11:58:29.833599 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application created_value = creator( May 19 11:58:29.834697 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 11:58:29.834697 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 11:58:29.834697 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 11:58:29.834697 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application May 19 11:58:29.836210 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b7cd77-998b-493d-8d46-11b66c338030 None None] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:29.837131 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 364/738] 10.222.0.23 () {56 vars in 1108 bytes} [Thu May 19 11:58:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 11:58:29.838798 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12af2d1c-8fb4-43c3-9105-df01b8dd3d18 None None] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:29.839091 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12af2d1c-8fb4-43c3-9105-df01b8dd3d18 None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:29.839372 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12af2d1c-8fb4-43c3-9105-df01b8dd3d18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:29.858524 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.auth.core [None req-12af2d1c-8fb4-43c3-9105-df01b8dd3d18 None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 11:58:29.859730 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-12af2d1c-8fb4-43c3-9105-df01b8dd3d18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 11:58:29.879051 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98709f0-58bd-4c99-8a64-a6b71a4b50f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:29.879634 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98709f0-58bd-4c99-8a64-a6b71a4b50f3 None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:29.879950 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98709f0-58bd-4c99-8a64-a6b71a4b50f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:29.880226 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98709f0-58bd-4c99-8a64-a6b71a4b50f3 None admin] PATH_INFO: `/v3/domains` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:29.880950 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98709f0-58bd-4c99-8a64-a6b71a4b50f3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 11:58:29.881258 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98709f0-58bd-4c99-8a64-a6b71a4b50f3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:29.882645 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98709f0-58bd-4c99-8a64-a6b71a4b50f3 None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:29.895915 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 375/739] 10.222.0.23 () {56 vars in 1109 bytes} [Thu May 19 11:58:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:58:29.897228 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29315a5e-644c-45c2-bf7f-901e39ae0476 None None] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:29.897524 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29315a5e-644c-45c2-bf7f-901e39ae0476 None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:29.897800 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29315a5e-644c-45c2-bf7f-901e39ae0476 None None] PATH_INFO: `/` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:29.903147 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 376/740] 10.222.0.23 () {54 vars in 926 bytes} [Thu May 19 11:58:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 11:58:29.903147 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29315a5e-644c-45c2-bf7f-901e39ae0476 None None] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:29.949984 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 365/741] 10.222.0.23 () {58 vars in 1023 bytes} [Thu May 19 11:58:29 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 11:58:29.950388 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12af2d1c-8fb4-43c3-9105-df01b8dd3d18 None None] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:29.995472 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:58:30.022139 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29315a5e-644c-45c2-bf7f-901e39ae0476 None None] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:30.030236 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:58:30.054579 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12af2d1c-8fb4-43c3-9105-df01b8dd3d18 None None] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:30.059918 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e3e2a3-3bad-4e27-b68e-2ae1bdf064d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:30.061459 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e3e2a3-3bad-4e27-b68e-2ae1bdf064d2 None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:30.061711 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e3e2a3-3bad-4e27-b68e-2ae1bdf064d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:30.061962 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e3e2a3-3bad-4e27-b68e-2ae1bdf064d2 None admin] PATH_INFO: `/v3/domains` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:30.062874 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e3e2a3-3bad-4e27-b68e-2ae1bdf064d2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 11:58:30.063196 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e3e2a3-3bad-4e27-b68e-2ae1bdf064d2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:30.064632 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e3e2a3-3bad-4e27-b68e-2ae1bdf064d2 None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:30.077474 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 377/742] 10.222.0.23 () {56 vars in 1109 bytes} [Thu May 19 11:58:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:58:30.078728 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e6ecad-f4e2-488b-97af-c8da890c133d None None] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:30.079016 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e6ecad-f4e2-488b-97af-c8da890c133d None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:30.079290 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e6ecad-f4e2-488b-97af-c8da890c133d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:30.096036 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.auth.core [None req-14e6ecad-f4e2-488b-97af-c8da890c133d None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 11:58:30.097223 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-14e6ecad-f4e2-488b-97af-c8da890c133d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 11:58:30.098342 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b50a3ad-67f3-49e4-8bf3-c9c5623188fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:30.099229 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b50a3ad-67f3-49e4-8bf3-c9c5623188fa None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:30.099586 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b50a3ad-67f3-49e4-8bf3-c9c5623188fa None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:30.099955 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b50a3ad-67f3-49e4-8bf3-c9c5623188fa None admin] PATH_INFO: `/v3/users/nova` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:30.108280 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b50a3ad-67f3-49e4-8bf3-c9c5623188fa None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:30.109879 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b50a3ad-67f3-49e4-8bf3-c9c5623188fa None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:30.124707 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application [None req-6b50a3ad-67f3-49e4-8bf3-c9c5623188fa None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 19 11:58:30.124707 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 11:58:30.124707 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 11:58:30.124707 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 11:58:30.124707 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 11:58:30.124707 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 11:58:30.124707 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 11:58:30.124707 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 11:58:30.124707 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 11:58:30.124707 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 11:58:30.124707 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 11:58:30.124707 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 11:58:30.124707 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 11:58:30.124707 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 11:58:30.124707 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 11:58:30.124707 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 11:58:30.125654 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 11:58:30.125654 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 11:58:30.125654 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 11:58:30.125654 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 11:58:30.125654 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 11:58:30.125654 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 11:58:30.125654 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 11:58:30.125654 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 11:58:30.125654 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 11:58:30.125654 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.get_or_create( May 19 11:58:30.125654 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 11:58:30.125654 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application with Lock( May 19 11:58:30.125654 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 11:58:30.125654 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._enter() May 19 11:58:30.125654 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 11:58:30.125654 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 11:58:30.125654 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 11:58:30.125654 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.creator() May 19 11:58:30.126584 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 11:58:30.126584 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application created_value = creator( May 19 11:58:30.126584 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 11:58:30.126584 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 11:58:30.126584 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 11:58:30.126584 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 11:58:30.126584 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 11:58:30.126584 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 11:58:30.126584 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 19 11:58:30.126584 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application May 19 11:58:30.126584 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 366/743] 10.222.0.23 () {56 vars in 1093 bytes} [Thu May 19 11:58:29 2022] GET /identity/v3/users/nova => generated 82 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 11:58:30.127153 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b50a3ad-67f3-49e4-8bf3-c9c5623188fa None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:30.178329 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 378/744] 10.222.0.23 () {58 vars in 1023 bytes} [Thu May 19 11:58:30 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 11:58:30.179564 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e6ecad-f4e2-488b-97af-c8da890c133d None None] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:30.251478 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b50a3ad-67f3-49e4-8bf3-c9c5623188fa None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:30.259223 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:58:30.291421 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dde6160-eb23-4c6d-b183-15f540d9268d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:30.292341 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dde6160-eb23-4c6d-b183-15f540d9268d None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:30.292626 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dde6160-eb23-4c6d-b183-15f540d9268d None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:30.292904 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dde6160-eb23-4c6d-b183-15f540d9268d None admin] PATH_INFO: `/v3/roles/service` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:30.294997 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:58:30.297686 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e6ecad-f4e2-488b-97af-c8da890c133d None None] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:30.304272 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dde6160-eb23-4c6d-b183-15f540d9268d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 11:58:30.312310 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application [None req-6dde6160-eb23-4c6d-b183-15f540d9268d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 11:58:30.312310 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 11:58:30.312310 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 11:58:30.312310 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 11:58:30.312310 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 11:58:30.312310 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 11:58:30.312310 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 11:58:30.312310 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 11:58:30.312310 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 11:58:30.312310 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 11:58:30.312310 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 11:58:30.312310 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 11:58:30.312310 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 11:58:30.312310 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 11:58:30.312310 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 11:58:30.312310 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 11:58:30.313251 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 11:58:30.313251 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 11:58:30.313251 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 11:58:30.313251 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 11:58:30.313251 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 11:58:30.313251 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 11:58:30.313251 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 11:58:30.313251 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 11:58:30.313251 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 11:58:30.313251 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.get_or_create( May 19 11:58:30.313251 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 11:58:30.313251 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application with Lock( May 19 11:58:30.313251 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 11:58:30.313251 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._enter() May 19 11:58:30.313251 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 11:58:30.313251 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 11:58:30.314069 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 11:58:30.314069 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.creator() May 19 11:58:30.314069 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 11:58:30.314069 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application created_value = creator( May 19 11:58:30.314069 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 11:58:30.314069 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 11:58:30.314069 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 11:58:30.314069 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 11:58:30.314069 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 11:58:30.314069 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 11:58:30.314069 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 11:58:30.314069 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application May 19 11:58:30.314069 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 367/745] 10.222.0.23 () {56 vars in 1102 bytes} [Thu May 19 11:58:30 2022] GET /identity/v3/roles/service => generated 85 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 11:58:30.314738 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dde6160-eb23-4c6d-b183-15f540d9268d None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:30.340791 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321b937d-1832-48b0-b32b-dc723c748a0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:30.341523 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321b937d-1832-48b0-b32b-dc723c748a0e None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:30.341858 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321b937d-1832-48b0-b32b-dc723c748a0e None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:30.342148 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321b937d-1832-48b0-b32b-dc723c748a0e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:30.350250 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321b937d-1832-48b0-b32b-dc723c748a0e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:30.351721 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321b937d-1832-48b0-b32b-dc723c748a0e None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:30.369266 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application [None req-321b937d-1832-48b0-b32b-dc723c748a0e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 11:58:30.369266 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 11:58:30.369266 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 11:58:30.369266 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 11:58:30.369266 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 11:58:30.369266 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 11:58:30.369266 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 11:58:30.369266 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 11:58:30.369266 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 11:58:30.369266 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 11:58:30.369266 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 11:58:30.369266 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 11:58:30.369266 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 11:58:30.369266 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 11:58:30.369266 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 11:58:30.370259 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 11:58:30.370259 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 11:58:30.370259 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 11:58:30.370259 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 11:58:30.370259 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 11:58:30.370259 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 11:58:30.370259 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.get_or_create( May 19 11:58:30.370259 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 11:58:30.370259 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application with Lock( May 19 11:58:30.370259 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 11:58:30.370259 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._enter() May 19 11:58:30.370259 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 11:58:30.370259 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 11:58:30.370259 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 11:58:30.370259 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.creator() May 19 11:58:30.370259 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 11:58:30.370259 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application created_value = creator( May 19 11:58:30.371177 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 11:58:30.371177 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 11:58:30.371177 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 11:58:30.371177 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 11:58:30.371177 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 11:58:30.371177 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 11:58:30.371177 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 11:58:30.371177 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application May 19 11:58:30.371177 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 379/746] 10.222.0.23 () {56 vars in 1111 bytes} [Thu May 19 11:58:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 11:58:30.371655 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321b937d-1832-48b0-b32b-dc723c748a0e None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:30.432285 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dde6160-eb23-4c6d-b183-15f540d9268d None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:30.468429 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37f0a03-5c01-42c9-80a0-40431d8a2d32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:30.469107 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37f0a03-5c01-42c9-80a0-40431d8a2d32 None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:30.469750 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37f0a03-5c01-42c9-80a0-40431d8a2d32 None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:30.469750 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37f0a03-5c01-42c9-80a0-40431d8a2d32 None admin] PATH_INFO: `/v3/users` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:30.470475 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37f0a03-5c01-42c9-80a0-40431d8a2d32 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 11:58:30.470779 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37f0a03-5c01-42c9-80a0-40431d8a2d32 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:30.472216 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37f0a03-5c01-42c9-80a0-40431d8a2d32 None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:30.483991 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321b937d-1832-48b0-b32b-dc723c748a0e None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:30.495737 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 368/747] 10.222.0.23 () {56 vars in 1133 bytes} [Thu May 19 11:58:30 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:58:30.495737 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37f0a03-5c01-42c9-80a0-40431d8a2d32 None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:30.513974 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:58:30.523376 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26cc3ffc-5fc3-4183-a307-d5777074880f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:30.524073 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cc3ffc-5fc3-4183-a307-d5777074880f None admin] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:30.524432 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cc3ffc-5fc3-4183-a307-d5777074880f None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:30.524814 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cc3ffc-5fc3-4183-a307-d5777074880f None admin] PATH_INFO: `/v3/services` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:30.525537 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26cc3ffc-5fc3-4183-a307-d5777074880f None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:30.527020 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26cc3ffc-5fc3-4183-a307-d5777074880f None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:30.551700 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:58:30.597482 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37f0a03-5c01-42c9-80a0-40431d8a2d32 None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:30.612614 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 380/748] 10.222.0.23 () {60 vars in 1141 bytes} [Thu May 19 11:58:30 2022] POST /identity/v3/services => generated 240 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 11:58:30.613661 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26cc3ffc-5fc3-4183-a307-d5777074880f None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:30.664765 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d126d302-78f2-4ebd-9ea4-b061dc7e9698 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:30.668506 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d126d302-78f2-4ebd-9ea4-b061dc7e9698 None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:30.669036 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d126d302-78f2-4ebd-9ea4-b061dc7e9698 None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:30.669476 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d126d302-78f2-4ebd-9ea4-b061dc7e9698 None admin] PATH_INFO: `/v3/roles` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:30.671736 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d126d302-78f2-4ebd-9ea4-b061dc7e9698 None admin] RBAC: Adding query filter params (name=service) {{(pid=80539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 11:58:30.672294 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d126d302-78f2-4ebd-9ea4-b061dc7e9698 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:30.673956 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d126d302-78f2-4ebd-9ea4-b061dc7e9698 None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:30.685118 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 369/749] 10.222.0.23 () {56 vars in 1103 bytes} [Thu May 19 11:58:30 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:58:30.687016 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d126d302-78f2-4ebd-9ea4-b061dc7e9698 None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:30.757143 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26cc3ffc-5fc3-4183-a307-d5777074880f None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:30.799628 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca792a66-3c27-4556-b9fe-c2a648edc022 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:30.800350 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca792a66-3c27-4556-b9fe-c2a648edc022 None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:30.800640 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca792a66-3c27-4556-b9fe-c2a648edc022 None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:30.800921 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca792a66-3c27-4556-b9fe-c2a648edc022 None admin] PATH_INFO: `/v3/projects` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:30.801735 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca792a66-3c27-4556-b9fe-c2a648edc022 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 11:58:30.802100 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca792a66-3c27-4556-b9fe-c2a648edc022 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:30.803602 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca792a66-3c27-4556-b9fe-c2a648edc022 None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:30.813531 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d126d302-78f2-4ebd-9ea4-b061dc7e9698 None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:30.822851 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 381/750] 10.222.0.23 () {56 vars in 1148 bytes} [Thu May 19 11:58:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:58:30.824046 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca792a66-3c27-4556-b9fe-c2a648edc022 None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:30.832932 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:58:30.864733 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8246d293-8db6-4407-9291-03633f5d339f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:30.864973 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8246d293-8db6-4407-9291-03633f5d339f None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:30.865330 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8246d293-8db6-4407-9291-03633f5d339f None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:30.866236 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8246d293-8db6-4407-9291-03633f5d339f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:30.878305 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8246d293-8db6-4407-9291-03633f5d339f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:30.880106 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8246d293-8db6-4407-9291-03633f5d339f None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:30.898982 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application [None req-8246d293-8db6-4407-9291-03633f5d339f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 11:58:30.898982 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 11:58:30.898982 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 11:58:30.898982 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 11:58:30.898982 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 11:58:30.898982 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 11:58:30.898982 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 11:58:30.898982 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 11:58:30.898982 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 11:58:30.898982 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 11:58:30.898982 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 11:58:30.898982 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 11:58:30.898982 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 11:58:30.898982 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 11:58:30.898982 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 11:58:30.899979 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 11:58:30.899979 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 11:58:30.899979 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 11:58:30.899979 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 11:58:30.899979 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 11:58:30.899979 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 11:58:30.899979 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.get_or_create( May 19 11:58:30.899979 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 11:58:30.899979 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application with Lock( May 19 11:58:30.899979 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 11:58:30.899979 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._enter() May 19 11:58:30.899979 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 11:58:30.899979 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 11:58:30.899979 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 11:58:30.899979 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.creator() May 19 11:58:30.899979 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 11:58:30.899979 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application created_value = creator( May 19 11:58:30.900866 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 11:58:30.900866 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 11:58:30.900866 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 11:58:30.900866 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application May 19 11:58:30.900866 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 370/751] 10.222.0.23 () {56 vars in 1108 bytes} [Thu May 19 11:58:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 11:58:30.902370 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8246d293-8db6-4407-9291-03633f5d339f None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:30.910869 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:58:30.987955 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca792a66-3c27-4556-b9fe-c2a648edc022 None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:31.027068 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8246d293-8db6-4407-9291-03633f5d339f None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:31.028899 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6524b728-b917-4740-b9cb-cf21e8eb7831 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:31.030737 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6524b728-b917-4740-b9cb-cf21e8eb7831 None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:31.031272 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6524b728-b917-4740-b9cb-cf21e8eb7831 None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:31.031762 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6524b728-b917-4740-b9cb-cf21e8eb7831 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:31.046457 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6524b728-b917-4740-b9cb-cf21e8eb7831 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:31.048512 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6524b728-b917-4740-b9cb-cf21e8eb7831 None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:31.067212 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b3b6a3-9979-4dee-82f0-7a3c6fa1c2ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:31.067941 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b3b6a3-9979-4dee-82f0-7a3c6fa1c2ce None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:31.068179 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b3b6a3-9979-4dee-82f0-7a3c6fa1c2ce None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:31.068453 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b3b6a3-9979-4dee-82f0-7a3c6fa1c2ce None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:31.069286 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b3b6a3-9979-4dee-82f0-7a3c6fa1c2ce None admin] RBAC: Adding query filter params (role.id=9db5671c65d3414ea2682afa2ccf1403, scope.project.id=d70d25bf69314820b8aaecb65576978a, user.id=15cbfc73ae1243d2a8b1e70f9cd99876) {{(pid=80539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 11:58:31.069597 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b3b6a3-9979-4dee-82f0-7a3c6fa1c2ce None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:31.070646 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application [None req-6524b728-b917-4740-b9cb-cf21e8eb7831 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 11:58:31.070646 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 11:58:31.070646 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 11:58:31.070646 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 11:58:31.070646 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 11:58:31.070646 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 11:58:31.070646 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 11:58:31.070646 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 11:58:31.070646 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 11:58:31.070646 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 11:58:31.070646 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 11:58:31.070646 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 11:58:31.070646 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 11:58:31.070646 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 11:58:31.070646 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 11:58:31.071475 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 11:58:31.071475 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 11:58:31.071475 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 11:58:31.071475 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 11:58:31.071475 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 11:58:31.071475 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 11:58:31.071475 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.get_or_create( May 19 11:58:31.071475 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 11:58:31.071475 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application with Lock( May 19 11:58:31.071475 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 11:58:31.071475 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._enter() May 19 11:58:31.071475 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 11:58:31.071475 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 11:58:31.071475 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 11:58:31.071475 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.creator() May 19 11:58:31.071475 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 11:58:31.071475 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application created_value = creator( May 19 11:58:31.072398 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 11:58:31.072398 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 11:58:31.072398 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 11:58:31.072398 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application May 19 11:58:31.072610 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b3b6a3-9979-4dee-82f0-7a3c6fa1c2ce None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:31.074858 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 382/752] 10.222.0.23 () {56 vars in 1108 bytes} [Thu May 19 11:58:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 11:58:31.075948 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6524b728-b917-4740-b9cb-cf21e8eb7831 None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:31.082414 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 371/753] 10.222.0.23 () {56 vars in 1374 bytes} [Thu May 19 11:58:30 2022] GET /identity/v3/role_assignments?user.id=15cbfc73ae1243d2a8b1e70f9cd99876&scope.project.id=d70d25bf69314820b8aaecb65576978a&role.id=9db5671c65d3414ea2682afa2ccf1403 => generated 259 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:58:31.095826 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b3b6a3-9979-4dee-82f0-7a3c6fa1c2ce None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:31.149677 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:58:31.196074 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:58:31.200755 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6524b728-b917-4740-b9cb-cf21e8eb7831 None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:31.273774 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b3b6a3-9979-4dee-82f0-7a3c6fa1c2ce None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:31.309177 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c358cd-e00e-4e76-b053-006ac5abc4e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:31.309867 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c358cd-e00e-4e76-b053-006ac5abc4e0 None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:31.310189 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c358cd-e00e-4e76-b053-006ac5abc4e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:31.310489 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c358cd-e00e-4e76-b053-006ac5abc4e0 None admin] PATH_INFO: `/v3/domains` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:31.311333 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c358cd-e00e-4e76-b053-006ac5abc4e0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 11:58:31.311620 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c358cd-e00e-4e76-b053-006ac5abc4e0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:31.313052 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c358cd-e00e-4e76-b053-006ac5abc4e0 None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:31.315911 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed2cbd8-8db3-4305-949f-cb5a9dc9fd4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:31.316930 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed2cbd8-8db3-4305-949f-cb5a9dc9fd4f None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:31.317415 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed2cbd8-8db3-4305-949f-cb5a9dc9fd4f None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:31.317888 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed2cbd8-8db3-4305-949f-cb5a9dc9fd4f None admin] PATH_INFO: `/v3/domains` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:31.318848 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed2cbd8-8db3-4305-949f-cb5a9dc9fd4f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 11:58:31.319370 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed2cbd8-8db3-4305-949f-cb5a9dc9fd4f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:31.321292 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed2cbd8-8db3-4305-949f-cb5a9dc9fd4f None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:31.336537 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 372/754] 10.222.0.23 () {56 vars in 1109 bytes} [Thu May 19 11:58:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:58:31.342356 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed2cbd8-8db3-4305-949f-cb5a9dc9fd4f None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:31.348041 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 383/755] 10.222.0.23 () {56 vars in 1109 bytes} [Thu May 19 11:58:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:58:31.353118 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c358cd-e00e-4e76-b053-006ac5abc4e0 None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:31.448439 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:58:31.465809 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed2cbd8-8db3-4305-949f-cb5a9dc9fd4f None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:31.486994 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c358cd-e00e-4e76-b053-006ac5abc4e0 None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:31.504543 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aea97f0-e96b-464c-94fd-9abe506a041e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:31.505359 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aea97f0-e96b-464c-94fd-9abe506a041e None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:31.505637 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aea97f0-e96b-464c-94fd-9abe506a041e None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:31.505911 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aea97f0-e96b-464c-94fd-9abe506a041e None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:31.515002 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aea97f0-e96b-464c-94fd-9abe506a041e None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:31.516663 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aea97f0-e96b-464c-94fd-9abe506a041e None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:31.533624 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application [None req-7aea97f0-e96b-464c-94fd-9abe506a041e None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 19 11:58:31.533624 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 11:58:31.533624 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 11:58:31.533624 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 11:58:31.533624 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 11:58:31.533624 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 11:58:31.533624 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 11:58:31.533624 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 11:58:31.533624 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 11:58:31.533624 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 11:58:31.533624 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 11:58:31.533624 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 11:58:31.533624 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 11:58:31.533624 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 11:58:31.533624 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 11:58:31.533624 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 11:58:31.534841 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 11:58:31.534841 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 11:58:31.534841 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 11:58:31.534841 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 11:58:31.534841 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 11:58:31.534841 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 11:58:31.534841 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 11:58:31.534841 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 11:58:31.534841 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 11:58:31.534841 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.get_or_create( May 19 11:58:31.534841 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 11:58:31.534841 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application with Lock( May 19 11:58:31.534841 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 11:58:31.534841 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._enter() May 19 11:58:31.534841 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 11:58:31.534841 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 11:58:31.534841 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 11:58:31.534841 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.creator() May 19 11:58:31.536068 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 11:58:31.536068 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application created_value = creator( May 19 11:58:31.536068 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 11:58:31.536068 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 11:58:31.536068 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 11:58:31.536068 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 11:58:31.536068 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 11:58:31.536068 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 11:58:31.536068 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 19 11:58:31.536068 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application May 19 11:58:31.539354 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 373/756] 10.222.0.23 () {56 vars in 1117 bytes} [Thu May 19 11:58:31 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 11:58:31.540198 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0839f398-ebe6-4d73-bc26-db8e66232102 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:31.540706 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c01d8b2-5745-4fa6-b468-fe4d088d43d0 None None] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:31.540995 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c01d8b2-5745-4fa6-b468-fe4d088d43d0 None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:31.541272 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c01d8b2-5745-4fa6-b468-fe4d088d43d0 None None] PATH_INFO: `/` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:31.541768 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0839f398-ebe6-4d73-bc26-db8e66232102 None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:31.542231 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0839f398-ebe6-4d73-bc26-db8e66232102 None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:31.542683 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0839f398-ebe6-4d73-bc26-db8e66232102 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:31.547618 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 374/757] 10.222.0.23 () {54 vars in 926 bytes} [Thu May 19 11:58:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 11:58:31.549165 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c01d8b2-5745-4fa6-b468-fe4d088d43d0 None None] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:31.552242 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0839f398-ebe6-4d73-bc26-db8e66232102 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:31.554948 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0839f398-ebe6-4d73-bc26-db8e66232102 None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:31.578651 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application [None req-0839f398-ebe6-4d73-bc26-db8e66232102 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 11:58:31.578651 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 11:58:31.578651 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 11:58:31.578651 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 11:58:31.578651 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 11:58:31.578651 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 11:58:31.578651 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 11:58:31.578651 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 11:58:31.578651 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 11:58:31.578651 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 11:58:31.578651 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 11:58:31.578651 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 11:58:31.578651 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 11:58:31.578651 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 11:58:31.578651 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 11:58:31.579613 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 11:58:31.579613 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 11:58:31.579613 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 11:58:31.579613 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 11:58:31.579613 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 11:58:31.579613 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 11:58:31.579613 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.get_or_create( May 19 11:58:31.579613 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 11:58:31.579613 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application with Lock( May 19 11:58:31.579613 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 11:58:31.579613 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._enter() May 19 11:58:31.579613 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 11:58:31.579613 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 11:58:31.579613 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 11:58:31.579613 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.creator() May 19 11:58:31.579613 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 11:58:31.579613 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application created_value = creator( May 19 11:58:31.580541 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 11:58:31.580541 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 11:58:31.580541 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 11:58:31.580541 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 11:58:31.580541 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 11:58:31.580541 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 11:58:31.580541 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 11:58:31.580541 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application May 19 11:58:31.586429 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 384/758] 10.222.0.23 () {56 vars in 1111 bytes} [Thu May 19 11:58:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 11:58:31.586429 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67414cd-034b-4dc2-ac8a-51d0637bf080 None None] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:31.586429 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67414cd-034b-4dc2-ac8a-51d0637bf080 None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:31.586429 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67414cd-034b-4dc2-ac8a-51d0637bf080 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:31.611817 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.auth.core [None req-a67414cd-034b-4dc2-ac8a-51d0637bf080 None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 11:58:31.614136 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-a67414cd-034b-4dc2-ac8a-51d0637bf080 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 11:58:31.632745 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:58:31.677938 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c01d8b2-5745-4fa6-b468-fe4d088d43d0 None None] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:31.699985 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 385/759] 10.222.0.23 () {58 vars in 1023 bytes} [Thu May 19 11:58:31 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 11:58:31.702032 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67414cd-034b-4dc2-ac8a-51d0637bf080 None None] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:31.723731 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a39317-0a84-498f-a741-4408b87d3975 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:31.724439 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a39317-0a84-498f-a741-4408b87d3975 None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:31.724721 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a39317-0a84-498f-a741-4408b87d3975 None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:31.724995 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a39317-0a84-498f-a741-4408b87d3975 None admin] PATH_INFO: `/v3/users` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:31.725849 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a39317-0a84-498f-a741-4408b87d3975 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 11:58:31.726162 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a39317-0a84-498f-a741-4408b87d3975 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:31.727602 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a39317-0a84-498f-a741-4408b87d3975 None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:31.759861 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 375/760] 10.222.0.23 () {56 vars in 1149 bytes} [Thu May 19 11:58:31 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:58:31.759861 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a39317-0a84-498f-a741-4408b87d3975 None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:31.766299 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:58:31.860056 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67414cd-034b-4dc2-ac8a-51d0637bf080 None None] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:31.871468 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a39317-0a84-498f-a741-4408b87d3975 None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:31.912519 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cfe32df-94e2-4a3d-bbd0-95fbd878914c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:31.913746 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a36311-0bc8-44a6-b4e3-f7843e5a2919 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:31.914570 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a36311-0bc8-44a6-b4e3-f7843e5a2919 None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:31.914836 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfe32df-94e2-4a3d-bbd0-95fbd878914c None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:31.915062 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a36311-0bc8-44a6-b4e3-f7843e5a2919 None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:31.915439 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a36311-0bc8-44a6-b4e3-f7843e5a2919 None admin] PATH_INFO: `/v3/projects` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:31.915561 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfe32df-94e2-4a3d-bbd0-95fbd878914c None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:31.916101 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfe32df-94e2-4a3d-bbd0-95fbd878914c None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:31.916551 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a36311-0bc8-44a6-b4e3-f7843e5a2919 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 11:58:31.916956 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a36311-0bc8-44a6-b4e3-f7843e5a2919 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:31.917221 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cfe32df-94e2-4a3d-bbd0-95fbd878914c None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:31.918713 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a36311-0bc8-44a6-b4e3-f7843e5a2919 None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:31.919007 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cfe32df-94e2-4a3d-bbd0-95fbd878914c None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:31.934383 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 386/761] 10.222.0.23 () {56 vars in 1148 bytes} [Thu May 19 11:58:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:58:31.935654 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a36311-0bc8-44a6-b4e3-f7843e5a2919 None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:31.938120 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application [None req-9cfe32df-94e2-4a3d-bbd0-95fbd878914c None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. May 19 11:58:31.938120 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 11:58:31.938120 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 11:58:31.938120 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 11:58:31.938120 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 11:58:31.938120 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 11:58:31.938120 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 11:58:31.938120 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 11:58:31.938120 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 19 11:58:31.938120 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application May 19 11:58:31.938120 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 11:58:31.938120 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application May 19 11:58:31.938120 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 11:58:31.938120 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 11:58:31.938120 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 11:58:31.938120 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 11:58:31.939256 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 11:58:31.939256 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 11:58:31.939256 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 11:58:31.939256 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 11:58:31.939256 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 11:58:31.939256 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 11:58:31.939256 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 11:58:31.939256 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 11:58:31.939256 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 11:58:31.939256 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 11:58:31.939256 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 11:58:31.939256 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 11:58:31.939256 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 11:58:31.939256 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 11:58:31.939256 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 11:58:31.939256 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 11:58:31.940200 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.get_or_create( May 19 11:58:31.940200 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 11:58:31.940200 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application with Lock( May 19 11:58:31.940200 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 11:58:31.940200 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._enter() May 19 11:58:31.940200 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 11:58:31.940200 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 11:58:31.940200 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 11:58:31.940200 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.creator() May 19 11:58:31.940200 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 11:58:31.940200 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application created_value = creator( May 19 11:58:31.940200 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 11:58:31.940200 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 11:58:31.940200 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 19 11:58:31.940200 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application May 19 11:58:31.942963 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 376/762] 10.222.0.23 () {56 vars in 1129 bytes} [Thu May 19 11:58:31 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 11:58:31.947222 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cfe32df-94e2-4a3d-bbd0-95fbd878914c None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:31.956201 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:58:32.023380 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:58:32.067870 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cfe32df-94e2-4a3d-bbd0-95fbd878914c None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:32.075276 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a36311-0bc8-44a6-b4e3-f7843e5a2919 None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:32.112678 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900c7a3f-7116-466e-8b3c-e9cc08974b4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:32.113685 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900c7a3f-7116-466e-8b3c-e9cc08974b4c None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:32.113815 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf709725-8e78-4b41-8069-993824cd6e0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:32.114353 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900c7a3f-7116-466e-8b3c-e9cc08974b4c None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:32.114685 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf709725-8e78-4b41-8069-993824cd6e0e None admin] REQUEST_METHOD: `PUT` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:32.114967 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf709725-8e78-4b41-8069-993824cd6e0e None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:32.115237 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf709725-8e78-4b41-8069-993824cd6e0e None admin] PATH_INFO: `/v3/projects/d70d25bf69314820b8aaecb65576978a/users/34a98d75d84b4c67981377e0f26eca62/roles/9db5671c65d3414ea2682afa2ccf1403` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:32.115553 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900c7a3f-7116-466e-8b3c-e9cc08974b4c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:32.117328 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf709725-8e78-4b41-8069-993824cd6e0e None admin] RBAC: Authorizing `identity:create_grant(project_id=d70d25bf69314820b8aaecb65576978a, user_id=34a98d75d84b4c67981377e0f26eca62, role_id=9db5671c65d3414ea2682afa2ccf1403)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:32.119048 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf709725-8e78-4b41-8069-993824cd6e0e None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:32.189810 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900c7a3f-7116-466e-8b3c-e9cc08974b4c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:32.190975 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 377/763] 10.222.0.23 () {58 vars in 1439 bytes} [Thu May 19 11:58:31 2022] PUT /identity/v3/projects/d70d25bf69314820b8aaecb65576978a/users/34a98d75d84b4c67981377e0f26eca62/roles/9db5671c65d3414ea2682afa2ccf1403 => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 11:58:32.192021 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf709725-8e78-4b41-8069-993824cd6e0e None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:32.192466 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900c7a3f-7116-466e-8b3c-e9cc08974b4c None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:32.205850 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:58:32.332942 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf709725-8e78-4b41-8069-993824cd6e0e None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:32.368180 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:58:32.379290 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58be887-1194-4ea1-8b03-37d3f0da3b37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:32.380413 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58be887-1194-4ea1-8b03-37d3f0da3b37 None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:32.381482 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58be887-1194-4ea1-8b03-37d3f0da3b37 None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:32.381976 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58be887-1194-4ea1-8b03-37d3f0da3b37 None admin] PATH_INFO: `/v3/services` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:32.382911 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58be887-1194-4ea1-8b03-37d3f0da3b37 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 11:58:32.383453 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58be887-1194-4ea1-8b03-37d3f0da3b37 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:32.385781 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58be887-1194-4ea1-8b03-37d3f0da3b37 None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:32.396733 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 378/764] 10.222.0.23 () {56 vars in 1124 bytes} [Thu May 19 11:58:32 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:58:32.399396 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58be887-1194-4ea1-8b03-37d3f0da3b37 None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:32.415645 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 387/765] 10.222.0.23 () {58 vars in 1282 bytes} [Thu May 19 11:58:31 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 480 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 11:58:32.421799 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900c7a3f-7116-466e-8b3c-e9cc08974b4c None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:32.573234 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900c7a3f-7116-466e-8b3c-e9cc08974b4c None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:32.579193 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58be887-1194-4ea1-8b03-37d3f0da3b37 None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:32.586351 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:58:32.623559 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84bf8d1-cad0-4333-86a9-77a842adb36f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:32.624770 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84bf8d1-cad0-4333-86a9-77a842adb36f None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:32.625279 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84bf8d1-cad0-4333-86a9-77a842adb36f None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:32.625751 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84bf8d1-cad0-4333-86a9-77a842adb36f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:32.641061 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3b0847-0275-45e3-aa72-539b4837b364 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:32.641628 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84bf8d1-cad0-4333-86a9-77a842adb36f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:32.641848 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3b0847-0275-45e3-aa72-539b4837b364 None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:32.642131 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3b0847-0275-45e3-aa72-539b4837b364 None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:32.642440 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3b0847-0275-45e3-aa72-539b4837b364 None admin] PATH_INFO: `/v3/services` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:32.643128 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84bf8d1-cad0-4333-86a9-77a842adb36f None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:32.643236 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3b0847-0275-45e3-aa72-539b4837b364 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 11:58:32.643460 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3b0847-0275-45e3-aa72-539b4837b364 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:32.645004 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3b0847-0275-45e3-aa72-539b4837b364 None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:32.651520 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 379/766] 10.222.0.23 () {56 vars in 1124 bytes} [Thu May 19 11:58:32 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:58:32.659312 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3b0847-0275-45e3-aa72-539b4837b364 None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:32.671405 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application [None req-c84bf8d1-cad0-4333-86a9-77a842adb36f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 11:58:32.671405 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 11:58:32.671405 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 11:58:32.671405 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 11:58:32.671405 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 11:58:32.671405 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 11:58:32.671405 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 11:58:32.671405 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 11:58:32.671405 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 11:58:32.671405 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 11:58:32.671405 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 11:58:32.671405 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 11:58:32.671405 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 11:58:32.671405 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 11:58:32.671405 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 11:58:32.672479 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 11:58:32.672479 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 11:58:32.672479 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 11:58:32.672479 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 11:58:32.672479 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 11:58:32.672479 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 11:58:32.672479 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.get_or_create( May 19 11:58:32.672479 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 11:58:32.672479 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application with Lock( May 19 11:58:32.672479 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 11:58:32.672479 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._enter() May 19 11:58:32.672479 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 11:58:32.672479 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 11:58:32.672479 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 11:58:32.672479 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.creator() May 19 11:58:32.672479 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 11:58:32.672479 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application created_value = creator( May 19 11:58:32.673338 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 11:58:32.673338 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 11:58:32.673338 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 11:58:32.673338 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application May 19 11:58:32.675667 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 388/767] 10.222.0.23 () {56 vars in 1108 bytes} [Thu May 19 11:58:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 11:58:32.681255 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84bf8d1-cad0-4333-86a9-77a842adb36f None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:32.730559 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:58:32.793145 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84bf8d1-cad0-4333-86a9-77a842adb36f None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:32.798033 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3b0847-0275-45e3-aa72-539b4837b364 None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:32.855731 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8a5726-d210-4f1d-afe4-6f565de23030 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:32.859418 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8a5726-d210-4f1d-afe4-6f565de23030 None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:32.860156 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8a5726-d210-4f1d-afe4-6f565de23030 None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:32.860768 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8a5726-d210-4f1d-afe4-6f565de23030 None admin] PATH_INFO: `/v3/domains` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:32.862283 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8a5726-d210-4f1d-afe4-6f565de23030 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 11:58:32.864176 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8a5726-d210-4f1d-afe4-6f565de23030 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:32.866264 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8a5726-d210-4f1d-afe4-6f565de23030 None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:32.869516 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21400628-8c7c-45ba-bd99-41ac3e45863b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:32.870459 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21400628-8c7c-45ba-bd99-41ac3e45863b None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:32.870956 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21400628-8c7c-45ba-bd99-41ac3e45863b None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:32.871429 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21400628-8c7c-45ba-bd99-41ac3e45863b None admin] PATH_INFO: `/v3/endpoints` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:32.872643 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21400628-8c7c-45ba-bd99-41ac3e45863b None admin] RBAC: Adding query filter params (interface=public, service_id=694cd9b0205747b4a4a16a19349f43ab, region_id=RegionOne) {{(pid=80539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 11:58:32.874519 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21400628-8c7c-45ba-bd99-41ac3e45863b None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:32.878594 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21400628-8c7c-45ba-bd99-41ac3e45863b None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:32.892917 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 389/768] 10.222.0.23 () {56 vars in 1109 bytes} [Thu May 19 11:58:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:58:32.897493 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 380/769] 10.222.0.23 () {56 vars in 1251 bytes} [Thu May 19 11:58:32 2022] GET /identity/v3/endpoints?service_id=694cd9b0205747b4a4a16a19349f43ab&interface=public®ion_id=RegionOne => generated 194 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:58:32.898614 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8a5726-d210-4f1d-afe4-6f565de23030 None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:32.901299 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21400628-8c7c-45ba-bd99-41ac3e45863b None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:32.936344 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:58:33.025893 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21400628-8c7c-45ba-bd99-41ac3e45863b None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:33.064264 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:58:33.082844 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8a5726-d210-4f1d-afe4-6f565de23030 None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:33.106266 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8803cd7c-c797-45c1-b354-76becb929f11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:33.106933 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8803cd7c-c797-45c1-b354-76becb929f11 None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:33.107505 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8803cd7c-c797-45c1-b354-76becb929f11 None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:33.108129 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8803cd7c-c797-45c1-b354-76becb929f11 None admin] PATH_INFO: `/v3/services` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:33.108845 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8803cd7c-c797-45c1-b354-76becb929f11 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:33.110241 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8803cd7c-c797-45c1-b354-76becb929f11 None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:33.120846 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 381/770] 10.222.0.23 () {56 vars in 1087 bytes} [Thu May 19 11:58:32 2022] GET /identity/v3/services => generated 772 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:58:33.146166 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e670046-8797-45b8-bc05-b9d92b7641b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:33.146903 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e670046-8797-45b8-bc05-b9d92b7641b9 None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:33.147191 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e670046-8797-45b8-bc05-b9d92b7641b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:33.147470 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e670046-8797-45b8-bc05-b9d92b7641b9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:33.156569 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e670046-8797-45b8-bc05-b9d92b7641b9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:33.158234 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e670046-8797-45b8-bc05-b9d92b7641b9 None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:33.174809 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application [None req-6e670046-8797-45b8-bc05-b9d92b7641b9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 11:58:33.174809 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 11:58:33.174809 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 11:58:33.174809 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 11:58:33.174809 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 11:58:33.174809 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 11:58:33.174809 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 11:58:33.174809 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 11:58:33.174809 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 11:58:33.174809 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 11:58:33.174809 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 11:58:33.174809 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 11:58:33.174809 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 11:58:33.174809 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 11:58:33.174809 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 11:58:33.175763 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 11:58:33.175763 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 11:58:33.175763 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 11:58:33.175763 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 11:58:33.175763 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 11:58:33.175763 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 11:58:33.175763 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.get_or_create( May 19 11:58:33.175763 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 11:58:33.175763 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application with Lock( May 19 11:58:33.175763 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 11:58:33.175763 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._enter() May 19 11:58:33.175763 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 11:58:33.175763 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 11:58:33.175763 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 11:58:33.175763 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.creator() May 19 11:58:33.175763 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 11:58:33.175763 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application created_value = creator( May 19 11:58:33.176671 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 11:58:33.176671 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 11:58:33.176671 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 11:58:33.176671 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 11:58:33.176671 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 11:58:33.176671 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 11:58:33.176671 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 11:58:33.176671 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application May 19 11:58:33.176671 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 390/771] 10.222.0.23 () {56 vars in 1111 bytes} [Thu May 19 11:58:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 11:58:33.182850 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8803cd7c-c797-45c1-b354-76becb929f11 None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:33.204108 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f39fb83-7cdc-48ac-b51d-d95ac50bf35e None None] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:33.204430 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f39fb83-7cdc-48ac-b51d-d95ac50bf35e None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:33.204681 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f39fb83-7cdc-48ac-b51d-d95ac50bf35e None None] PATH_INFO: `/` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:33.206578 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 391/772] 10.222.0.23 () {54 vars in 926 bytes} [Thu May 19 11:58:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 11:58:33.212898 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d1792f-768f-4b58-8343-b421cedd1a38 None None] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:33.213200 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d1792f-768f-4b58-8343-b421cedd1a38 None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:33.213466 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d1792f-768f-4b58-8343-b421cedd1a38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:33.235273 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.auth.core [None req-36d1792f-768f-4b58-8343-b421cedd1a38 None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 11:58:33.237217 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-36d1792f-768f-4b58-8343-b421cedd1a38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 11:58:33.309569 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8803cd7c-c797-45c1-b354-76becb929f11 None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:33.364078 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f1fb8a-513f-47ae-a5b5-abc6e31b757d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:33.364764 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f1fb8a-513f-47ae-a5b5-abc6e31b757d None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:33.365051 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f1fb8a-513f-47ae-a5b5-abc6e31b757d None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:33.365331 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f1fb8a-513f-47ae-a5b5-abc6e31b757d None admin] PATH_INFO: `/v3/projects` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:33.366139 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f1fb8a-513f-47ae-a5b5-abc6e31b757d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 11:58:33.366450 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f1fb8a-513f-47ae-a5b5-abc6e31b757d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:33.368205 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f1fb8a-513f-47ae-a5b5-abc6e31b757d None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:33.394210 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 382/773] 10.222.0.23 () {56 vars in 1148 bytes} [Thu May 19 11:58:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 11:58:33.395820 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 392/774] 10.222.0.23 () {58 vars in 1023 bytes} [Thu May 19 11:58:33 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 11:58:33.400228 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f1fb8a-513f-47ae-a5b5-abc6e31b757d None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:33.431769 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d1792f-768f-4b58-8343-b421cedd1a38 None None] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:33.446831 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:58:33.470276 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:58:33.539402 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f1fb8a-513f-47ae-a5b5-abc6e31b757d None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:33.582010 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d1792f-768f-4b58-8343-b421cedd1a38 None None] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:33.583742 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80299f89-b3a0-4686-871f-bda0b1ab387e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:33.584725 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80299f89-b3a0-4686-871f-bda0b1ab387e None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:33.585119 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80299f89-b3a0-4686-871f-bda0b1ab387e None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:33.585492 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80299f89-b3a0-4686-871f-bda0b1ab387e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:33.586523 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80299f89-b3a0-4686-871f-bda0b1ab387e None admin] RBAC: Adding query filter params (role.id=7c59061ba7064a0db6032433fb0eec6b, scope.project.id=d70d25bf69314820b8aaecb65576978a, user.id=0c3181fb4f584a549a786adfefdf3dfb) {{(pid=80539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 11:58:33.586985 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80299f89-b3a0-4686-871f-bda0b1ab387e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:33.588609 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80299f89-b3a0-4686-871f-bda0b1ab387e None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:33.598080 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 383/775] 10.222.0.23 () {56 vars in 1374 bytes} [Thu May 19 11:58:33 2022] GET /identity/v3/role_assignments?user.id=0c3181fb4f584a549a786adfefdf3dfb&scope.project.id=d70d25bf69314820b8aaecb65576978a&role.id=7c59061ba7064a0db6032433fb0eec6b => generated 259 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 11:58:33.632294 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6cbd7e-1086-4d77-8e3f-747cb4370193 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:33.633373 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6cbd7e-1086-4d77-8e3f-747cb4370193 None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:33.633831 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6cbd7e-1086-4d77-8e3f-747cb4370193 None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:33.634259 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6cbd7e-1086-4d77-8e3f-747cb4370193 None admin] PATH_INFO: `/v3/services/network` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:33.635070 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6cbd7e-1086-4d77-8e3f-747cb4370193 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:33.636670 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6cbd7e-1086-4d77-8e3f-747cb4370193 None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:33.651300 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application [None req-7e6cbd7e-1086-4d77-8e3f-747cb4370193 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. May 19 11:58:33.651300 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 11:58:33.651300 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 11:58:33.651300 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 11:58:33.651300 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 11:58:33.651300 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 11:58:33.651300 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 11:58:33.651300 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 11:58:33.651300 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 19 11:58:33.651300 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application May 19 11:58:33.651300 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 11:58:33.651300 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application May 19 11:58:33.651300 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 11:58:33.651300 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 11:58:33.651300 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 11:58:33.651300 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 11:58:33.651300 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 11:58:33.652550 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 11:58:33.652550 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 11:58:33.652550 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 11:58:33.652550 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 11:58:33.652550 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 11:58:33.652550 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 11:58:33.652550 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 11:58:33.652550 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 11:58:33.652550 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 11:58:33.652550 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 11:58:33.652550 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 11:58:33.652550 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 11:58:33.652550 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 11:58:33.652550 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 11:58:33.652550 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 11:58:33.652550 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.get_or_create( May 19 11:58:33.653574 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 11:58:33.653574 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application with Lock( May 19 11:58:33.653574 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 11:58:33.653574 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._enter() May 19 11:58:33.653574 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 11:58:33.653574 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 11:58:33.653574 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 11:58:33.653574 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.creator() May 19 11:58:33.653574 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 11:58:33.653574 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application created_value = creator( May 19 11:58:33.653574 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 11:58:33.653574 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 11:58:33.653574 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 19 11:58:33.653574 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application May 19 11:58:33.657604 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 393/776] 10.222.0.23 () {56 vars in 1111 bytes} [Thu May 19 11:58:33 2022] GET /identity/v3/services/network => generated 88 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 11:58:33.663826 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80299f89-b3a0-4686-871f-bda0b1ab387e None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:33.791842 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80299f89-b3a0-4686-871f-bda0b1ab387e None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:33.804642 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:58:33.844379 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81233786-841c-41bd-b155-b10da7973f17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:33.847582 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81233786-841c-41bd-b155-b10da7973f17 None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:33.847905 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81233786-841c-41bd-b155-b10da7973f17 None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:33.848168 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81233786-841c-41bd-b155-b10da7973f17 None admin] PATH_INFO: `/v3/services` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:33.848908 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81233786-841c-41bd-b155-b10da7973f17 None admin] RBAC: Adding query filter params (name=network) {{(pid=80539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 11:58:33.849218 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81233786-841c-41bd-b155-b10da7973f17 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:33.850755 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81233786-841c-41bd-b155-b10da7973f17 None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:33.861469 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 384/777] 10.222.0.23 () {56 vars in 1112 bytes} [Thu May 19 11:58:33 2022] GET /identity/v3/services?name=network => generated 124 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:58:33.867561 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6cbd7e-1086-4d77-8e3f-747cb4370193 None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:33.873746 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd69e70-d717-4555-8056-914f1dcfbfd9 None None] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:33.873943 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd69e70-d717-4555-8056-914f1dcfbfd9 None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:33.874197 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd69e70-d717-4555-8056-914f1dcfbfd9 None None] PATH_INFO: `/` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:33.875828 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 385/778] 10.222.0.23 () {54 vars in 926 bytes} [Thu May 19 11:58:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 11:58:33.887815 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112a103a-9e8a-4d70-b1cc-61a64cee73e2 None None] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:33.887815 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112a103a-9e8a-4d70-b1cc-61a64cee73e2 None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:33.887815 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112a103a-9e8a-4d70-b1cc-61a64cee73e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:33.910301 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.auth.core [None req-112a103a-9e8a-4d70-b1cc-61a64cee73e2 None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 11:58:33.911419 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-112a103a-9e8a-4d70-b1cc-61a64cee73e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 11:58:34.030344 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:58:34.056378 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 386/779] 10.222.0.23 () {58 vars in 1023 bytes} [Thu May 19 11:58:33 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 11:58:34.059447 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6cbd7e-1086-4d77-8e3f-747cb4370193 None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:34.092057 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112a103a-9e8a-4d70-b1cc-61a64cee73e2 None None] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:34.102387 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe415da3-000f-42b8-a028-8c2e9d9c57df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:34.103071 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe415da3-000f-42b8-a028-8c2e9d9c57df None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:34.103360 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe415da3-000f-42b8-a028-8c2e9d9c57df None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:34.103643 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe415da3-000f-42b8-a028-8c2e9d9c57df None admin] PATH_INFO: `/v3/services` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:34.104434 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe415da3-000f-42b8-a028-8c2e9d9c57df None admin] RBAC: Adding query filter params (type=network) {{(pid=80540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 11:58:34.104749 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe415da3-000f-42b8-a028-8c2e9d9c57df None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:34.106150 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe415da3-000f-42b8-a028-8c2e9d9c57df None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:34.119589 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 394/780] 10.222.0.23 () {56 vars in 1112 bytes} [Thu May 19 11:58:33 2022] GET /identity/v3/services?type=network => generated 350 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:58:34.125247 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe415da3-000f-42b8-a028-8c2e9d9c57df None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:34.270203 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112a103a-9e8a-4d70-b1cc-61a64cee73e2 None None] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:34.276124 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:58:34.322845 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe415da3-000f-42b8-a028-8c2e9d9c57df None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:34.328870 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dec2877-dda2-430d-bdaa-e96ffd58c5fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:34.329920 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dec2877-dda2-430d-bdaa-e96ffd58c5fb None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:34.331250 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dec2877-dda2-430d-bdaa-e96ffd58c5fb None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:34.332316 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dec2877-dda2-430d-bdaa-e96ffd58c5fb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:34.355069 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dec2877-dda2-430d-bdaa-e96ffd58c5fb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 11:58:34.376001 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application [None req-1dec2877-dda2-430d-bdaa-e96ffd58c5fb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 11:58:34.376001 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 11:58:34.376001 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 11:58:34.376001 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 11:58:34.376001 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 11:58:34.376001 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 11:58:34.376001 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 11:58:34.376001 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 11:58:34.376001 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 11:58:34.376001 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 11:58:34.376001 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 11:58:34.376001 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 11:58:34.376001 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 11:58:34.376001 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 11:58:34.376001 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 11:58:34.376001 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 11:58:34.377089 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 11:58:34.377089 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 11:58:34.377089 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 11:58:34.377089 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 11:58:34.377089 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 11:58:34.377089 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 11:58:34.377089 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 11:58:34.377089 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 11:58:34.377089 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 11:58:34.377089 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.get_or_create( May 19 11:58:34.377089 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 11:58:34.377089 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application with Lock( May 19 11:58:34.377089 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 11:58:34.377089 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._enter() May 19 11:58:34.377089 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 11:58:34.377089 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 11:58:34.377959 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 11:58:34.377959 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.creator() May 19 11:58:34.377959 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 11:58:34.377959 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application created_value = creator( May 19 11:58:34.377959 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 11:58:34.377959 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 11:58:34.377959 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 11:58:34.377959 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 11:58:34.377959 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 11:58:34.377959 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 11:58:34.377959 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 11:58:34.377959 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application May 19 11:58:34.382799 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 387/781] 10.222.0.23 () {56 vars in 1096 bytes} [Thu May 19 11:58:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 291 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 11:58:34.385050 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00c898d-9250-4e89-8fb8-c01675c13b4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:34.385968 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00c898d-9250-4e89-8fb8-c01675c13b4b None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:34.386520 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00c898d-9250-4e89-8fb8-c01675c13b4b None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:34.387585 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dec2877-dda2-430d-bdaa-e96ffd58c5fb None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:34.388944 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00c898d-9250-4e89-8fb8-c01675c13b4b None admin] PATH_INFO: `/v3/endpoints` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:34.390926 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00c898d-9250-4e89-8fb8-c01675c13b4b None admin] RBAC: Adding query filter params (interface=public, service_id=24ec3fba543746d6a5dc2d020e11beea, region_id=RegionOne) {{(pid=80540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 11:58:34.393085 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00c898d-9250-4e89-8fb8-c01675c13b4b None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:34.395586 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00c898d-9250-4e89-8fb8-c01675c13b4b None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:34.412676 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 395/782] 10.222.0.23 () {56 vars in 1251 bytes} [Thu May 19 11:58:34 2022] GET /identity/v3/endpoints?service_id=24ec3fba543746d6a5dc2d020e11beea&interface=public®ion_id=RegionOne => generated 194 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:58:34.419474 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00c898d-9250-4e89-8fb8-c01675c13b4b None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:34.427977 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:58:34.526736 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dec2877-dda2-430d-bdaa-e96ffd58c5fb None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:34.590895 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aadcd63a-574b-4231-a818-994dcd2183e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:34.592626 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadcd63a-574b-4231-a818-994dcd2183e2 None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:34.593145 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadcd63a-574b-4231-a818-994dcd2183e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:34.593644 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadcd63a-574b-4231-a818-994dcd2183e2 None admin] PATH_INFO: `/v3/roles` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:34.594623 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aadcd63a-574b-4231-a818-994dcd2183e2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 11:58:34.596887 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aadcd63a-574b-4231-a818-994dcd2183e2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:34.598535 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aadcd63a-574b-4231-a818-994dcd2183e2 None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:34.614376 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 388/783] 10.222.0.23 () {56 vars in 1099 bytes} [Thu May 19 11:58:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:58:34.621534 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aadcd63a-574b-4231-a818-994dcd2183e2 None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:34.652380 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00c898d-9250-4e89-8fb8-c01675c13b4b None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:34.676200 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:58:34.708463 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746b2c75-7723-4f41-ae3f-7812ec556950 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:34.709566 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746b2c75-7723-4f41-ae3f-7812ec556950 None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:34.710072 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746b2c75-7723-4f41-ae3f-7812ec556950 None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:34.711572 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746b2c75-7723-4f41-ae3f-7812ec556950 None admin] PATH_INFO: `/v3/services` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:34.712520 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746b2c75-7723-4f41-ae3f-7812ec556950 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:34.720411 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746b2c75-7723-4f41-ae3f-7812ec556950 None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:34.722630 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 396/784] 10.222.0.23 () {56 vars in 1087 bytes} [Thu May 19 11:58:34 2022] GET /identity/v3/services => generated 772 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:58:34.788297 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aadcd63a-574b-4231-a818-994dcd2183e2 None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:34.829139 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee9020a-9fa3-43e1-9e5b-29ff923669c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:34.829972 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee9020a-9fa3-43e1-9e5b-29ff923669c4 None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:34.830261 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee9020a-9fa3-43e1-9e5b-29ff923669c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:34.830541 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee9020a-9fa3-43e1-9e5b-29ff923669c4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:34.845114 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee9020a-9fa3-43e1-9e5b-29ff923669c4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:34.846651 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee9020a-9fa3-43e1-9e5b-29ff923669c4 None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:34.867156 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application [None req-5ee9020a-9fa3-43e1-9e5b-29ff923669c4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 11:58:34.867156 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 11:58:34.867156 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 11:58:34.867156 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 11:58:34.867156 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 11:58:34.867156 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 11:58:34.867156 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 11:58:34.867156 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 11:58:34.867156 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 11:58:34.867156 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 11:58:34.867156 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 11:58:34.867156 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 11:58:34.867156 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 11:58:34.867156 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 11:58:34.867156 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 11:58:34.868183 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 11:58:34.868183 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 11:58:34.868183 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 11:58:34.868183 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 11:58:34.868183 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 11:58:34.868183 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 11:58:34.868183 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.get_or_create( May 19 11:58:34.868183 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 11:58:34.868183 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application with Lock( May 19 11:58:34.868183 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 11:58:34.868183 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._enter() May 19 11:58:34.868183 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 11:58:34.868183 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 11:58:34.868183 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 11:58:34.868183 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.creator() May 19 11:58:34.868183 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 11:58:34.868183 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application created_value = creator( May 19 11:58:34.869640 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 11:58:34.869640 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 11:58:34.869640 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 11:58:34.869640 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application May 19 11:58:34.869640 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 389/785] 10.222.0.23 () {56 vars in 1108 bytes} [Thu May 19 11:58:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 11:58:34.877117 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746b2c75-7723-4f41-ae3f-7812ec556950 None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:35.002006 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:58:35.034572 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746b2c75-7723-4f41-ae3f-7812ec556950 None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:35.046362 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bb61fd-74ad-4a22-8b4a-482fa5446034 None None] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:35.046362 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bb61fd-74ad-4a22-8b4a-482fa5446034 None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:35.046362 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bb61fd-74ad-4a22-8b4a-482fa5446034 None None] PATH_INFO: `/` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:35.046698 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 390/786] 10.222.0.23 () {54 vars in 926 bytes} [Thu May 19 11:58:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 11:58:35.057320 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46219a72-2075-4c2b-a4d8-acf75da98645 None None] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:35.057687 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46219a72-2075-4c2b-a4d8-acf75da98645 None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:35.057978 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46219a72-2075-4c2b-a4d8-acf75da98645 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:35.078492 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ac6921-e8ef-42a7-93d7-0be707816f48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:35.079173 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ac6921-e8ef-42a7-93d7-0be707816f48 None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:35.079458 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ac6921-e8ef-42a7-93d7-0be707816f48 None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:35.079739 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ac6921-e8ef-42a7-93d7-0be707816f48 None admin] PATH_INFO: `/v3/domains` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:35.080596 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ac6921-e8ef-42a7-93d7-0be707816f48 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 11:58:35.081051 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ac6921-e8ef-42a7-93d7-0be707816f48 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:35.082480 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ac6921-e8ef-42a7-93d7-0be707816f48 None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:35.088928 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.auth.core [None req-46219a72-2075-4c2b-a4d8-acf75da98645 None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 11:58:35.090154 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-46219a72-2075-4c2b-a4d8-acf75da98645 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 11:58:35.095741 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 397/787] 10.222.0.23 () {56 vars in 1109 bytes} [Thu May 19 11:58:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:58:35.104910 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ac6921-e8ef-42a7-93d7-0be707816f48 None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:35.213498 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 391/788] 10.222.0.23 () {58 vars in 1023 bytes} [Thu May 19 11:58:35 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 11:58:35.241297 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46219a72-2075-4c2b-a4d8-acf75da98645 None None] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:35.250490 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:58:35.274725 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ac6921-e8ef-42a7-93d7-0be707816f48 None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:35.330578 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:58:35.346761 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c1d096-3416-4c47-a5f0-a90868e5f0ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:35.347556 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c1d096-3416-4c47-a5f0-a90868e5f0ea None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:35.347863 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c1d096-3416-4c47-a5f0-a90868e5f0ea None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:35.348138 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c1d096-3416-4c47-a5f0-a90868e5f0ea None admin] PATH_INFO: `/v3/users/swift` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:35.360230 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c1d096-3416-4c47-a5f0-a90868e5f0ea None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:35.361696 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c1d096-3416-4c47-a5f0-a90868e5f0ea None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:35.389102 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application [None req-e1c1d096-3416-4c47-a5f0-a90868e5f0ea None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 19 11:58:35.389102 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 11:58:35.389102 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 11:58:35.389102 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 11:58:35.389102 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 11:58:35.389102 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 11:58:35.389102 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 11:58:35.389102 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 11:58:35.389102 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 11:58:35.389102 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 11:58:35.389102 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 11:58:35.389102 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 11:58:35.389102 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 11:58:35.389102 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 11:58:35.389102 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 11:58:35.389102 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 11:58:35.390141 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 11:58:35.390141 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 11:58:35.390141 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 11:58:35.390141 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 11:58:35.390141 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 11:58:35.390141 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 11:58:35.390141 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 11:58:35.390141 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 11:58:35.390141 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 11:58:35.390141 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.get_or_create( May 19 11:58:35.390141 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 11:58:35.390141 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application with Lock( May 19 11:58:35.390141 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 11:58:35.390141 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._enter() May 19 11:58:35.390141 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 11:58:35.390141 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 11:58:35.390141 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 11:58:35.390141 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.creator() May 19 11:58:35.391697 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 11:58:35.391697 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application created_value = creator( May 19 11:58:35.391697 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 11:58:35.391697 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 11:58:35.391697 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 11:58:35.391697 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 11:58:35.391697 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 11:58:35.391697 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 11:58:35.391697 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 19 11:58:35.391697 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application May 19 11:58:35.391697 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 398/789] 10.222.0.23 () {56 vars in 1096 bytes} [Thu May 19 11:58:35 2022] GET /identity/v3/users/swift => generated 83 bytes in 286 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 11:58:35.396720 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c1d096-3416-4c47-a5f0-a90868e5f0ea None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:35.457849 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46219a72-2075-4c2b-a4d8-acf75da98645 None None] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:35.522237 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c247b517-7686-4669-b7c2-75c0753b7fb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:35.526159 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c247b517-7686-4669-b7c2-75c0753b7fb3 None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:35.526639 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c247b517-7686-4669-b7c2-75c0753b7fb3 None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:35.527163 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c247b517-7686-4669-b7c2-75c0753b7fb3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:35.546764 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c247b517-7686-4669-b7c2-75c0753b7fb3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 11:58:35.555415 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application [None req-c247b517-7686-4669-b7c2-75c0753b7fb3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 11:58:35.555415 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 11:58:35.555415 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 11:58:35.555415 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 11:58:35.555415 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 11:58:35.555415 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 11:58:35.555415 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 11:58:35.555415 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 11:58:35.555415 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 11:58:35.555415 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 11:58:35.555415 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 11:58:35.555415 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 11:58:35.555415 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 11:58:35.555415 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 11:58:35.555415 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 11:58:35.555415 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 11:58:35.556589 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 11:58:35.556589 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 11:58:35.556589 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 11:58:35.556589 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 11:58:35.556589 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 11:58:35.556589 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 11:58:35.556589 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 11:58:35.556589 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 11:58:35.556589 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 11:58:35.556589 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.get_or_create( May 19 11:58:35.556589 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 11:58:35.556589 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application with Lock( May 19 11:58:35.556589 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 11:58:35.556589 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._enter() May 19 11:58:35.556589 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 11:58:35.556589 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 11:58:35.557601 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 11:58:35.557601 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.creator() May 19 11:58:35.557601 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 11:58:35.557601 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application created_value = creator( May 19 11:58:35.557601 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 11:58:35.557601 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 11:58:35.557601 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 11:58:35.557601 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 11:58:35.557601 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 11:58:35.557601 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 11:58:35.557601 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 11:58:35.557601 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application May 19 11:58:35.562098 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c1d096-3416-4c47-a5f0-a90868e5f0ea None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:35.576758 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 392/790] 10.222.0.23 () {56 vars in 1096 bytes} [Thu May 19 11:58:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 336 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 11:58:35.578885 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c247b517-7686-4669-b7c2-75c0753b7fb3 None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:35.640355 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833aba47-5360-43e0-97c1-8272fc6f448b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:35.641025 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833aba47-5360-43e0-97c1-8272fc6f448b None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:35.641312 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833aba47-5360-43e0-97c1-8272fc6f448b None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:35.641593 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833aba47-5360-43e0-97c1-8272fc6f448b None admin] PATH_INFO: `/v3/users` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:35.642448 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833aba47-5360-43e0-97c1-8272fc6f448b None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 11:58:35.642769 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833aba47-5360-43e0-97c1-8272fc6f448b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:35.644331 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833aba47-5360-43e0-97c1-8272fc6f448b None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:35.681686 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 399/791] 10.222.0.23 () {56 vars in 1135 bytes} [Thu May 19 11:58:35 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 11:58:35.688934 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833aba47-5360-43e0-97c1-8272fc6f448b None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:35.705552 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:58:35.771390 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:58:35.778381 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c247b517-7686-4669-b7c2-75c0753b7fb3 None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:35.845088 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1c6544-a048-4f6c-916d-51113e17176e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:35.845770 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1c6544-a048-4f6c-916d-51113e17176e None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:35.846060 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1c6544-a048-4f6c-916d-51113e17176e None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:35.846338 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1c6544-a048-4f6c-916d-51113e17176e None admin] PATH_INFO: `/v3/roles` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:35.847104 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1c6544-a048-4f6c-916d-51113e17176e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 11:58:35.847416 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1c6544-a048-4f6c-916d-51113e17176e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:35.848828 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1c6544-a048-4f6c-916d-51113e17176e None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:35.871945 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 393/792] 10.222.0.23 () {56 vars in 1099 bytes} [Thu May 19 11:58:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 11:58:35.875732 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1c6544-a048-4f6c-916d-51113e17176e None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:35.889539 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833aba47-5360-43e0-97c1-8272fc6f448b None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:35.928540 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21ff401-f3f5-44af-807f-880234727bc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:35.929324 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21ff401-f3f5-44af-807f-880234727bc6 None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:35.929621 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21ff401-f3f5-44af-807f-880234727bc6 None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:35.929908 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21ff401-f3f5-44af-807f-880234727bc6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:35.942535 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21ff401-f3f5-44af-807f-880234727bc6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:35.944054 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21ff401-f3f5-44af-807f-880234727bc6 None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:35.963824 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application [None req-a21ff401-f3f5-44af-807f-880234727bc6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 11:58:35.963824 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 11:58:35.963824 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 11:58:35.963824 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 11:58:35.963824 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 11:58:35.963824 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 11:58:35.963824 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 11:58:35.963824 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 11:58:35.963824 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 11:58:35.963824 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 11:58:35.963824 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 11:58:35.963824 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 11:58:35.963824 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 11:58:35.963824 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 11:58:35.963824 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 11:58:35.964760 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 11:58:35.964760 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 11:58:35.964760 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 11:58:35.964760 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 11:58:35.964760 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 11:58:35.964760 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 11:58:35.964760 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.get_or_create( May 19 11:58:35.964760 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 11:58:35.964760 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application with Lock( May 19 11:58:35.964760 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 11:58:35.964760 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._enter() May 19 11:58:35.964760 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 11:58:35.964760 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 11:58:35.964760 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 11:58:35.964760 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.creator() May 19 11:58:35.964760 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 11:58:35.964760 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application created_value = creator( May 19 11:58:35.965911 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 11:58:35.965911 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 11:58:35.965911 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 11:58:35.965911 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application May 19 11:58:35.965911 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 400/793] 10.222.0.23 () {56 vars in 1108 bytes} [Thu May 19 11:58:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 11:58:35.971320 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21ff401-f3f5-44af-807f-880234727bc6 None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:36.073462 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1c6544-a048-4f6c-916d-51113e17176e None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:36.107711 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:58:36.124197 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca450ca-641c-4d66-ab56-0d65fa2046eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:36.125254 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca450ca-641c-4d66-ab56-0d65fa2046eb None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:36.125695 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca450ca-641c-4d66-ab56-0d65fa2046eb None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:36.127378 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca450ca-641c-4d66-ab56-0d65fa2046eb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:36.154017 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca450ca-641c-4d66-ab56-0d65fa2046eb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:36.155537 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca450ca-641c-4d66-ab56-0d65fa2046eb None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:36.189629 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:58:36.197051 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21ff401-f3f5-44af-807f-880234727bc6 None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:36.200752 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application [None req-6ca450ca-641c-4d66-ab56-0d65fa2046eb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 11:58:36.200752 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 11:58:36.200752 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 11:58:36.200752 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 11:58:36.200752 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 11:58:36.200752 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 11:58:36.200752 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 11:58:36.200752 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 11:58:36.200752 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 11:58:36.200752 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 11:58:36.200752 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 11:58:36.200752 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 11:58:36.200752 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 11:58:36.200752 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 11:58:36.200752 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 11:58:36.207587 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 11:58:36.207587 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 11:58:36.207587 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 11:58:36.207587 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 11:58:36.207587 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 11:58:36.207587 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 11:58:36.207587 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.get_or_create( May 19 11:58:36.207587 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 11:58:36.207587 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application with Lock( May 19 11:58:36.207587 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 11:58:36.207587 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._enter() May 19 11:58:36.207587 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 11:58:36.207587 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 11:58:36.207587 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 11:58:36.207587 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.creator() May 19 11:58:36.207587 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 11:58:36.207587 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application created_value = creator( May 19 11:58:36.208677 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 11:58:36.208677 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 11:58:36.208677 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 11:58:36.208677 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application May 19 11:58:36.208677 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 394/794] 10.222.0.23 () {56 vars in 1108 bytes} [Thu May 19 11:58:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 327 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 11:58:36.209209 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca450ca-641c-4d66-ab56-0d65fa2046eb None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:36.242228 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73276f8-884f-46d7-bdf2-ce4f1369b7a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:36.244248 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73276f8-884f-46d7-bdf2-ce4f1369b7a3 None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:36.246272 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73276f8-884f-46d7-bdf2-ce4f1369b7a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:36.246771 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73276f8-884f-46d7-bdf2-ce4f1369b7a3 None admin] PATH_INFO: `/v3/domains` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:36.250197 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73276f8-884f-46d7-bdf2-ce4f1369b7a3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 11:58:36.250693 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73276f8-884f-46d7-bdf2-ce4f1369b7a3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:36.252326 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73276f8-884f-46d7-bdf2-ce4f1369b7a3 None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:36.278846 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 401/795] 10.222.0.23 () {56 vars in 1109 bytes} [Thu May 19 11:58:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:58:36.278846 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80769c7e-ff85-47eb-86d3-23968cc2d858 None None] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:36.278846 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80769c7e-ff85-47eb-86d3-23968cc2d858 None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:36.278846 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80769c7e-ff85-47eb-86d3-23968cc2d858 None None] PATH_INFO: `/` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:36.278846 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 402/796] 10.222.0.23 () {54 vars in 926 bytes} [Thu May 19 11:58:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 11:58:36.285756 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80769c7e-ff85-47eb-86d3-23968cc2d858 None None] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:36.359227 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca450ca-641c-4d66-ab56-0d65fa2046eb None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:36.401331 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80769c7e-ff85-47eb-86d3-23968cc2d858 None None] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:36.405938 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce01663e-bef5-4c25-8ac4-0fb1e29db89f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:36.406611 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce01663e-bef5-4c25-8ac4-0fb1e29db89f None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:36.406898 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce01663e-bef5-4c25-8ac4-0fb1e29db89f None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:36.407176 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce01663e-bef5-4c25-8ac4-0fb1e29db89f None admin] PATH_INFO: `/v3/domains` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:36.407941 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce01663e-bef5-4c25-8ac4-0fb1e29db89f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 11:58:36.408252 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce01663e-bef5-4c25-8ac4-0fb1e29db89f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:36.409709 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce01663e-bef5-4c25-8ac4-0fb1e29db89f None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:36.428246 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 395/797] 10.222.0.23 () {56 vars in 1109 bytes} [Thu May 19 11:58:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 11:58:36.428246 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0670c8-818b-4ee8-af4d-38906cecc7b6 None None] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:36.428246 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0670c8-818b-4ee8-af4d-38906cecc7b6 None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:36.428246 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0670c8-818b-4ee8-af4d-38906cecc7b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:36.438768 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ecf7709-492e-4c04-ade6-b13400f81ce9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:36.439577 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecf7709-492e-4c04-ade6-b13400f81ce9 None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:36.439952 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecf7709-492e-4c04-ade6-b13400f81ce9 None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:36.440177 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecf7709-492e-4c04-ade6-b13400f81ce9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:36.446353 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.auth.core [None req-7f0670c8-818b-4ee8-af4d-38906cecc7b6 None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 11:58:36.447343 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ecf7709-492e-4c04-ade6-b13400f81ce9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:36.448034 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-7f0670c8-818b-4ee8-af4d-38906cecc7b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 11:58:36.448849 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ecf7709-492e-4c04-ade6-b13400f81ce9 None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:36.462867 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application [None req-0ecf7709-492e-4c04-ade6-b13400f81ce9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 11:58:36.462867 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 11:58:36.462867 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 11:58:36.462867 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 11:58:36.462867 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 11:58:36.462867 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 11:58:36.462867 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 11:58:36.462867 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 11:58:36.462867 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 11:58:36.462867 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 11:58:36.462867 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 11:58:36.462867 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 11:58:36.462867 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 11:58:36.462867 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 11:58:36.462867 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 11:58:36.463826 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 11:58:36.463826 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 11:58:36.463826 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 11:58:36.463826 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 11:58:36.463826 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 11:58:36.463826 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 11:58:36.463826 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.get_or_create( May 19 11:58:36.463826 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 11:58:36.463826 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application with Lock( May 19 11:58:36.463826 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 11:58:36.463826 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._enter() May 19 11:58:36.463826 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 11:58:36.463826 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 11:58:36.463826 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 11:58:36.463826 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.creator() May 19 11:58:36.463826 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 11:58:36.463826 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application created_value = creator( May 19 11:58:36.464736 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 11:58:36.464736 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 11:58:36.464736 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 11:58:36.464736 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 11:58:36.464736 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 11:58:36.464736 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 11:58:36.464736 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 11:58:36.464736 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application May 19 11:58:36.467943 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 403/798] 10.222.0.23 () {56 vars in 1111 bytes} [Thu May 19 11:58:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 11:58:36.469878 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6c2d2c-daf4-4c87-a700-547e20d1de7e None None] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:36.470169 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6c2d2c-daf4-4c87-a700-547e20d1de7e None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:36.470459 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6c2d2c-daf4-4c87-a700-547e20d1de7e None None] PATH_INFO: `/` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:36.475897 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 404/799] 10.222.0.23 () {54 vars in 926 bytes} [Thu May 19 11:58:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 11:58:36.475897 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6c2d2c-daf4-4c87-a700-547e20d1de7e None None] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:36.485973 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:58:36.496054 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:58:36.531813 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 396/800] 10.222.0.23 () {58 vars in 1023 bytes} [Thu May 19 11:58:36 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 11:58:36.531813 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0670c8-818b-4ee8-af4d-38906cecc7b6 None None] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:36.590359 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6c2d2c-daf4-4c87-a700-547e20d1de7e None None] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:36.628739 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9289fadc-ed42-4e6c-b04b-296627c9eb10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:36.629582 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9289fadc-ed42-4e6c-b04b-296627c9eb10 None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:36.629870 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9289fadc-ed42-4e6c-b04b-296627c9eb10 None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:36.630149 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9289fadc-ed42-4e6c-b04b-296627c9eb10 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:36.635236 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0670c8-818b-4ee8-af4d-38906cecc7b6 None None] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:36.637567 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9289fadc-ed42-4e6c-b04b-296627c9eb10 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:36.639065 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9289fadc-ed42-4e6c-b04b-296627c9eb10 None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:36.657119 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application [None req-9289fadc-ed42-4e6c-b04b-296627c9eb10 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 19 11:58:36.657119 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 11:58:36.657119 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 11:58:36.657119 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 11:58:36.657119 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 11:58:36.657119 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 11:58:36.657119 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 11:58:36.657119 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 11:58:36.657119 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 11:58:36.657119 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 11:58:36.657119 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 11:58:36.657119 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 11:58:36.657119 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 11:58:36.657119 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 11:58:36.657119 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 11:58:36.657119 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 11:58:36.658304 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 11:58:36.658304 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 11:58:36.658304 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 11:58:36.658304 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 11:58:36.658304 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 11:58:36.658304 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 11:58:36.658304 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 11:58:36.658304 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 11:58:36.658304 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 11:58:36.658304 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.get_or_create( May 19 11:58:36.658304 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 11:58:36.658304 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application with Lock( May 19 11:58:36.658304 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 11:58:36.658304 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._enter() May 19 11:58:36.658304 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 11:58:36.658304 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 11:58:36.658304 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 11:58:36.658304 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.creator() May 19 11:58:36.659598 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 11:58:36.659598 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application created_value = creator( May 19 11:58:36.659598 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 11:58:36.659598 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 11:58:36.659598 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 11:58:36.659598 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 11:58:36.659598 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 11:58:36.659598 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 11:58:36.659598 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 19 11:58:36.659598 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application May 19 11:58:36.662823 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 405/801] 10.222.0.23 () {56 vars in 1093 bytes} [Thu May 19 11:58:36 2022] GET /identity/v3/users/nova => generated 82 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 11:58:36.664143 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202c0aa3-8ab0-440b-bbf7-cd19bacbfc2e None None] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:36.664440 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202c0aa3-8ab0-440b-bbf7-cd19bacbfc2e None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:36.664718 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202c0aa3-8ab0-440b-bbf7-cd19bacbfc2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:36.678208 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7b4e3f-c8df-4954-ad69-476ecc69c97b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:36.679197 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7b4e3f-c8df-4954-ad69-476ecc69c97b None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:36.679665 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7b4e3f-c8df-4954-ad69-476ecc69c97b None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:36.682148 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7b4e3f-c8df-4954-ad69-476ecc69c97b None admin] PATH_INFO: `/v3/projects` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:36.682726 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.auth.core [None req-202c0aa3-8ab0-440b-bbf7-cd19bacbfc2e None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 11:58:36.684291 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7b4e3f-c8df-4954-ad69-476ecc69c97b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 11:58:36.684404 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-202c0aa3-8ab0-440b-bbf7-cd19bacbfc2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 11:58:36.684897 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7b4e3f-c8df-4954-ad69-476ecc69c97b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:36.689530 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7b4e3f-c8df-4954-ad69-476ecc69c97b None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:36.715535 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 397/802] 10.222.0.23 () {56 vars in 1148 bytes} [Thu May 19 11:58:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:58:36.715535 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7b4e3f-c8df-4954-ad69-476ecc69c97b None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:36.762083 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:58:36.783185 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 406/803] 10.222.0.23 () {58 vars in 1023 bytes} [Thu May 19 11:58:36 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 11:58:36.784087 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202c0aa3-8ab0-440b-bbf7-cd19bacbfc2e None None] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:36.799903 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:58:36.854302 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7b4e3f-c8df-4954-ad69-476ecc69c97b None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:36.891119 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad83b121-a973-4ef5-ad55-86f4de8fd877 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:36.892206 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad83b121-a973-4ef5-ad55-86f4de8fd877 None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:36.892502 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad83b121-a973-4ef5-ad55-86f4de8fd877 None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:36.892785 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad83b121-a973-4ef5-ad55-86f4de8fd877 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:36.897509 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202c0aa3-8ab0-440b-bbf7-cd19bacbfc2e None None] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:36.904899 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad83b121-a973-4ef5-ad55-86f4de8fd877 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 11:58:36.913961 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application [None req-ad83b121-a973-4ef5-ad55-86f4de8fd877 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 11:58:36.913961 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 11:58:36.913961 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 11:58:36.913961 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 11:58:36.913961 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 11:58:36.913961 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 11:58:36.913961 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 11:58:36.913961 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 11:58:36.913961 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 11:58:36.913961 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 11:58:36.913961 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 11:58:36.913961 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 11:58:36.913961 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 11:58:36.913961 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 11:58:36.913961 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 11:58:36.913961 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 11:58:36.915767 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 11:58:36.915767 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 11:58:36.915767 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 11:58:36.915767 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 11:58:36.915767 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 11:58:36.915767 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 11:58:36.915767 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 11:58:36.915767 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 11:58:36.915767 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 11:58:36.915767 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.get_or_create( May 19 11:58:36.915767 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 11:58:36.915767 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application with Lock( May 19 11:58:36.915767 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 11:58:36.915767 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._enter() May 19 11:58:36.915767 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 11:58:36.915767 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 11:58:36.917309 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 11:58:36.917309 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.creator() May 19 11:58:36.917309 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 11:58:36.917309 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application created_value = creator( May 19 11:58:36.917309 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 11:58:36.917309 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 11:58:36.917309 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 11:58:36.917309 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 11:58:36.917309 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 11:58:36.917309 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 11:58:36.917309 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 11:58:36.917309 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application May 19 11:58:36.917309 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 398/804] 10.222.0.23 () {56 vars in 1102 bytes} [Thu May 19 11:58:36 2022] GET /identity/v3/roles/service => generated 85 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 11:58:36.917309 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad83b121-a973-4ef5-ad55-86f4de8fd877 None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:36.948280 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477f66c4-858a-4657-9719-5b8d6e010748 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:36.949729 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477f66c4-858a-4657-9719-5b8d6e010748 None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:36.949729 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477f66c4-858a-4657-9719-5b8d6e010748 None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:36.949909 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477f66c4-858a-4657-9719-5b8d6e010748 None admin] PATH_INFO: `/v3/users` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:36.950727 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477f66c4-858a-4657-9719-5b8d6e010748 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 11:58:36.951933 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477f66c4-858a-4657-9719-5b8d6e010748 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:36.953301 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477f66c4-858a-4657-9719-5b8d6e010748 None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:36.977415 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 407/805] 10.222.0.23 () {56 vars in 1133 bytes} [Thu May 19 11:58:36 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:58:36.977415 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477f66c4-858a-4657-9719-5b8d6e010748 None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:37.029842 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:58:37.040810 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad83b121-a973-4ef5-ad55-86f4de8fd877 None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:37.047522 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:58:37.082334 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d96e12-71c5-4de9-829c-5549a563efc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:37.083472 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d96e12-71c5-4de9-829c-5549a563efc2 None admin] REQUEST_METHOD: `PUT` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:37.083754 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d96e12-71c5-4de9-829c-5549a563efc2 None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:37.084375 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d96e12-71c5-4de9-829c-5549a563efc2 None admin] PATH_INFO: `/v3/projects/d70d25bf69314820b8aaecb65576978a/users/15cbfc73ae1243d2a8b1e70f9cd99876/roles/9db5671c65d3414ea2682afa2ccf1403` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:37.086933 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d96e12-71c5-4de9-829c-5549a563efc2 None admin] RBAC: Authorizing `identity:create_grant(project_id=d70d25bf69314820b8aaecb65576978a, user_id=15cbfc73ae1243d2a8b1e70f9cd99876, role_id=9db5671c65d3414ea2682afa2ccf1403)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:37.087678 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477f66c4-858a-4657-9719-5b8d6e010748 None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:37.088551 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d96e12-71c5-4de9-829c-5549a563efc2 None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:37.125247 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97648d63-f524-42e2-82ec-2cf1b9211a71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:37.126090 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97648d63-f524-42e2-82ec-2cf1b9211a71 None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:37.126376 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97648d63-f524-42e2-82ec-2cf1b9211a71 None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:37.126654 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97648d63-f524-42e2-82ec-2cf1b9211a71 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:37.127326 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97648d63-f524-42e2-82ec-2cf1b9211a71 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:37.129004 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97648d63-f524-42e2-82ec-2cf1b9211a71 None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:37.142880 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application [None req-97648d63-f524-42e2-82ec-2cf1b9211a71 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. May 19 11:58:37.142880 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 11:58:37.142880 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 11:58:37.142880 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 11:58:37.142880 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 11:58:37.142880 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 11:58:37.142880 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 11:58:37.142880 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 11:58:37.142880 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 19 11:58:37.142880 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application May 19 11:58:37.142880 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 11:58:37.142880 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application May 19 11:58:37.142880 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 11:58:37.142880 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 11:58:37.142880 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 11:58:37.142880 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 11:58:37.150778 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 11:58:37.150778 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 11:58:37.150778 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 11:58:37.150778 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 11:58:37.150778 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 11:58:37.150778 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 11:58:37.150778 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 11:58:37.150778 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 11:58:37.150778 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 11:58:37.150778 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 11:58:37.150778 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 11:58:37.150778 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 11:58:37.150778 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 11:58:37.150778 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 11:58:37.150778 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 11:58:37.150778 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 11:58:37.151758 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.get_or_create( May 19 11:58:37.151758 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 11:58:37.151758 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application with Lock( May 19 11:58:37.151758 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 11:58:37.151758 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._enter() May 19 11:58:37.151758 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 11:58:37.151758 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 11:58:37.151758 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 11:58:37.151758 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.creator() May 19 11:58:37.151758 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 11:58:37.151758 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application created_value = creator( May 19 11:58:37.151758 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 11:58:37.151758 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 11:58:37.151758 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 19 11:58:37.151758 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application May 19 11:58:37.151758 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 408/806] 10.222.0.23 () {56 vars in 1129 bytes} [Thu May 19 11:58:36 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 11:58:37.152853 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97648d63-f524-42e2-82ec-2cf1b9211a71 None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:37.170190 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 399/807] 10.222.0.23 () {58 vars in 1439 bytes} [Thu May 19 11:58:36 2022] PUT /identity/v3/projects/d70d25bf69314820b8aaecb65576978a/users/15cbfc73ae1243d2a8b1e70f9cd99876/roles/9db5671c65d3414ea2682afa2ccf1403 => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 11:58:37.171347 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d96e12-71c5-4de9-829c-5549a563efc2 None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:37.294678 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d96e12-71c5-4de9-829c-5549a563efc2 None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:37.312526 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97648d63-f524-42e2-82ec-2cf1b9211a71 None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:37.335711 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2fb8fc3-9cb6-422c-9193-c1d1e825acee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:37.336414 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fb8fc3-9cb6-422c-9193-c1d1e825acee None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:37.336704 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fb8fc3-9cb6-422c-9193-c1d1e825acee None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:37.336984 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fb8fc3-9cb6-422c-9193-c1d1e825acee None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:37.347268 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:58:37.356555 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f89ddab-c448-43ba-b26e-762b964ce6b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:37.362319 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f89ddab-c448-43ba-b26e-762b964ce6b4 None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:37.362319 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f89ddab-c448-43ba-b26e-762b964ce6b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:37.362319 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f89ddab-c448-43ba-b26e-762b964ce6b4 None admin] PATH_INFO: `/v3/roles` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:37.362319 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f89ddab-c448-43ba-b26e-762b964ce6b4 None admin] RBAC: Adding query filter params (name=service) {{(pid=80540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 11:58:37.362319 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f89ddab-c448-43ba-b26e-762b964ce6b4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:37.362319 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f89ddab-c448-43ba-b26e-762b964ce6b4 None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:37.365568 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:58:37.375014 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 409/808] 10.222.0.23 () {56 vars in 1103 bytes} [Thu May 19 11:58:37 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:58:37.376245 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f89ddab-c448-43ba-b26e-762b964ce6b4 None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:37.418926 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2fb8fc3-9cb6-422c-9193-c1d1e825acee None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:37.420877 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2fb8fc3-9cb6-422c-9193-c1d1e825acee None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:37.487716 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f89ddab-c448-43ba-b26e-762b964ce6b4 None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:37.527622 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68dc4fd-5ced-4025-bfc7-0ff57492fe40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:37.529423 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68dc4fd-5ced-4025-bfc7-0ff57492fe40 None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:37.530071 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68dc4fd-5ced-4025-bfc7-0ff57492fe40 None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:37.530666 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68dc4fd-5ced-4025-bfc7-0ff57492fe40 None admin] PATH_INFO: `/v3/services` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:37.532184 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68dc4fd-5ced-4025-bfc7-0ff57492fe40 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 11:58:37.532698 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68dc4fd-5ced-4025-bfc7-0ff57492fe40 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:37.534297 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68dc4fd-5ced-4025-bfc7-0ff57492fe40 None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:37.534472 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 400/809] 10.222.0.23 () {58 vars in 1282 bytes} [Thu May 19 11:58:37 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 11:58:37.535813 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edaf0b4-243b-4d67-923f-a6d339bad30b None None] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:37.536110 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edaf0b4-243b-4d67-923f-a6d339bad30b None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:37.536386 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edaf0b4-243b-4d67-923f-a6d339bad30b None None] PATH_INFO: `/` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:37.540438 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 401/810] 10.222.0.23 () {54 vars in 926 bytes} [Thu May 19 11:58:37 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 11:58:37.541579 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3edaf0b4-243b-4d67-923f-a6d339bad30b None None] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:37.548062 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 410/811] 10.222.0.23 () {56 vars in 1124 bytes} [Thu May 19 11:58:37 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:58:37.548062 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68dc4fd-5ced-4025-bfc7-0ff57492fe40 None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:37.598980 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:58:37.607568 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:58:37.653471 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3edaf0b4-243b-4d67-923f-a6d339bad30b None None] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:37.656086 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68dc4fd-5ced-4025-bfc7-0ff57492fe40 None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:37.693788 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a24b10d-28cf-48bc-b2e7-2776521b1ae3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:37.694335 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9caf6311-8e05-48d9-a206-a0bda8ac4c07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:37.694859 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a24b10d-28cf-48bc-b2e7-2776521b1ae3 None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:37.695127 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9caf6311-8e05-48d9-a206-a0bda8ac4c07 None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:37.695233 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a24b10d-28cf-48bc-b2e7-2776521b1ae3 None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:37.695404 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9caf6311-8e05-48d9-a206-a0bda8ac4c07 None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:37.695746 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a24b10d-28cf-48bc-b2e7-2776521b1ae3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:37.695964 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9caf6311-8e05-48d9-a206-a0bda8ac4c07 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:37.708119 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9caf6311-8e05-48d9-a206-a0bda8ac4c07 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:37.711119 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9caf6311-8e05-48d9-a206-a0bda8ac4c07 None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:37.711218 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a24b10d-28cf-48bc-b2e7-2776521b1ae3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:37.712198 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a24b10d-28cf-48bc-b2e7-2776521b1ae3 None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:37.729513 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application [None req-9caf6311-8e05-48d9-a206-a0bda8ac4c07 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 11:58:37.729513 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 11:58:37.729513 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 11:58:37.729513 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 11:58:37.729513 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 11:58:37.729513 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 11:58:37.729513 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 11:58:37.729513 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 11:58:37.729513 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 11:58:37.729513 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 11:58:37.729513 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 11:58:37.729513 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 11:58:37.729513 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 11:58:37.729513 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 11:58:37.729513 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 11:58:37.730441 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 11:58:37.730441 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 11:58:37.730441 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 11:58:37.730441 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 11:58:37.730441 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 11:58:37.730441 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 11:58:37.730441 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.get_or_create( May 19 11:58:37.730441 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 11:58:37.730441 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application with Lock( May 19 11:58:37.730441 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 11:58:37.730441 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._enter() May 19 11:58:37.730441 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 11:58:37.730441 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 11:58:37.730441 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 11:58:37.730441 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.creator() May 19 11:58:37.730441 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 11:58:37.730441 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application created_value = creator( May 19 11:58:37.736882 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 11:58:37.736882 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 11:58:37.736882 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 11:58:37.736882 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application May 19 11:58:37.737116 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application [None req-5a24b10d-28cf-48bc-b2e7-2776521b1ae3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 11:58:37.737116 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 11:58:37.737116 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 11:58:37.737116 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 11:58:37.737116 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 11:58:37.737116 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 11:58:37.737116 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 11:58:37.737116 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 11:58:37.737116 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 11:58:37.737116 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 11:58:37.737116 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 11:58:37.737116 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 11:58:37.737116 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 11:58:37.737116 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 11:58:37.737116 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 11:58:37.737901 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 11:58:37.737901 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 11:58:37.737901 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 11:58:37.737901 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 11:58:37.737901 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 11:58:37.737901 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 11:58:37.737901 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.get_or_create( May 19 11:58:37.737901 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 11:58:37.737901 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application with Lock( May 19 11:58:37.737901 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 11:58:37.737901 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._enter() May 19 11:58:37.737901 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 11:58:37.737901 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 11:58:37.737901 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 11:58:37.737901 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.creator() May 19 11:58:37.737901 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 11:58:37.737901 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application created_value = creator( May 19 11:58:37.739658 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 11:58:37.739658 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 11:58:37.739658 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 11:58:37.739658 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application May 19 11:58:37.740093 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 402/812] 10.222.0.23 () {56 vars in 1108 bytes} [Thu May 19 11:58:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 11:58:37.745464 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 411/813] 10.222.0.23 () {56 vars in 1108 bytes} [Thu May 19 11:58:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 11:58:37.745464 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a24b10d-28cf-48bc-b2e7-2776521b1ae3 None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:37.745663 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d56b8a-f24e-4fd1-a11c-4b3cf47e3a5c None None] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:37.745663 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d56b8a-f24e-4fd1-a11c-4b3cf47e3a5c None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:37.745663 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d56b8a-f24e-4fd1-a11c-4b3cf47e3a5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:37.754781 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.auth.core [None req-80d56b8a-f24e-4fd1-a11c-4b3cf47e3a5c None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 11:58:37.756018 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-80d56b8a-f24e-4fd1-a11c-4b3cf47e3a5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 11:58:37.837371 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 403/814] 10.222.0.23 () {58 vars in 1023 bytes} [Thu May 19 11:58:37 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 11:58:37.837371 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d56b8a-f24e-4fd1-a11c-4b3cf47e3a5c None None] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:37.850234 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a24b10d-28cf-48bc-b2e7-2776521b1ae3 None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:37.859473 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:58:37.874465 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:58:37.887270 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14236dad-8d5f-4d74-85a1-9ac91654ad45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:37.887993 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14236dad-8d5f-4d74-85a1-9ac91654ad45 None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:37.888328 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14236dad-8d5f-4d74-85a1-9ac91654ad45 None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:37.888621 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14236dad-8d5f-4d74-85a1-9ac91654ad45 None admin] PATH_INFO: `/v3/services` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:37.889418 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14236dad-8d5f-4d74-85a1-9ac91654ad45 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 11:58:37.890175 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14236dad-8d5f-4d74-85a1-9ac91654ad45 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:37.892395 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14236dad-8d5f-4d74-85a1-9ac91654ad45 None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:37.897615 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 412/815] 10.222.0.23 () {56 vars in 1124 bytes} [Thu May 19 11:58:37 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:58:37.911598 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14236dad-8d5f-4d74-85a1-9ac91654ad45 None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:37.956320 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d56b8a-f24e-4fd1-a11c-4b3cf47e3a5c None None] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:37.997329 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6291ad1d-468c-4888-933f-fa50c311ee8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:37.998399 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6291ad1d-468c-4888-933f-fa50c311ee8b None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:37.998773 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6291ad1d-468c-4888-933f-fa50c311ee8b None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:37.999062 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6291ad1d-468c-4888-933f-fa50c311ee8b None admin] PATH_INFO: `/v3/domains` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:37.999811 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6291ad1d-468c-4888-933f-fa50c311ee8b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 11:58:38.000648 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6291ad1d-468c-4888-933f-fa50c311ee8b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:38.002084 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6291ad1d-468c-4888-933f-fa50c311ee8b None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:38.012672 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14236dad-8d5f-4d74-85a1-9ac91654ad45 None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:38.016414 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 404/816] 10.222.0.23 () {56 vars in 1109 bytes} [Thu May 19 11:58:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:58:38.016736 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6291ad1d-468c-4888-933f-fa50c311ee8b None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:38.055927 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46397e5c-74c3-4545-badd-42050535e3c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:38.056592 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46397e5c-74c3-4545-badd-42050535e3c4 None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:38.056876 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46397e5c-74c3-4545-badd-42050535e3c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:38.057150 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46397e5c-74c3-4545-badd-42050535e3c4 None admin] PATH_INFO: `/v3/domains` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:38.057927 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46397e5c-74c3-4545-badd-42050535e3c4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 11:58:38.058242 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46397e5c-74c3-4545-badd-42050535e3c4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:38.059625 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46397e5c-74c3-4545-badd-42050535e3c4 None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:38.072696 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 413/817] 10.222.0.23 () {56 vars in 1109 bytes} [Thu May 19 11:58:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:58:38.073493 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46397e5c-74c3-4545-badd-42050535e3c4 None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:38.107747 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:58:38.116062 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:58:38.125431 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6291ad1d-468c-4888-933f-fa50c311ee8b None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:38.161915 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537d9692-bae1-4839-8988-172e2a1c876d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:38.162967 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537d9692-bae1-4839-8988-172e2a1c876d None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:38.163349 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537d9692-bae1-4839-8988-172e2a1c876d None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:38.163563 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537d9692-bae1-4839-8988-172e2a1c876d None admin] PATH_INFO: `/v3/services/network` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:38.164227 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537d9692-bae1-4839-8988-172e2a1c876d None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:38.165602 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537d9692-bae1-4839-8988-172e2a1c876d None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:38.178694 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application [None req-537d9692-bae1-4839-8988-172e2a1c876d None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. May 19 11:58:38.178694 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 11:58:38.178694 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 11:58:38.178694 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 11:58:38.178694 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 11:58:38.178694 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 11:58:38.178694 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 11:58:38.178694 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 11:58:38.178694 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 19 11:58:38.178694 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application May 19 11:58:38.178694 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 11:58:38.178694 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application May 19 11:58:38.178694 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 11:58:38.178694 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 11:58:38.178694 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 11:58:38.178694 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 11:58:38.178694 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 11:58:38.179939 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 11:58:38.179939 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 11:58:38.179939 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 11:58:38.179939 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 11:58:38.179939 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 11:58:38.179939 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 11:58:38.179939 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 11:58:38.179939 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 11:58:38.179939 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 11:58:38.179939 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 11:58:38.179939 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 11:58:38.179939 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 11:58:38.179939 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 11:58:38.179939 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 11:58:38.179939 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 11:58:38.179939 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.get_or_create( May 19 11:58:38.180961 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 11:58:38.180961 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application with Lock( May 19 11:58:38.180961 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 11:58:38.180961 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._enter() May 19 11:58:38.180961 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 11:58:38.180961 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 11:58:38.180961 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 11:58:38.180961 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.creator() May 19 11:58:38.180961 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 11:58:38.180961 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application created_value = creator( May 19 11:58:38.180961 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 11:58:38.180961 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 11:58:38.180961 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 19 11:58:38.180961 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application May 19 11:58:38.185542 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 405/818] 10.222.0.23 () {56 vars in 1111 bytes} [Thu May 19 11:58:38 2022] GET /identity/v3/services/network => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 11:58:38.186709 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537d9692-bae1-4839-8988-172e2a1c876d None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:38.187894 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46397e5c-74c3-4545-badd-42050535e3c4 None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:38.227261 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca6692e-2e71-444e-9c85-a7d40a443d43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:38.228118 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca6692e-2e71-444e-9c85-a7d40a443d43 None admin] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:38.228509 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca6692e-2e71-444e-9c85-a7d40a443d43 None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:38.228891 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca6692e-2e71-444e-9c85-a7d40a443d43 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:38.229735 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca6692e-2e71-444e-9c85-a7d40a443d43 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:38.231233 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca6692e-2e71-444e-9c85-a7d40a443d43 None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:38.291037 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537d9692-bae1-4839-8988-172e2a1c876d None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:38.326762 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247264ad-9fe8-490f-a6e9-8ea434e05aa8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:38.327777 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247264ad-9fe8-490f-a6e9-8ea434e05aa8 None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:38.328162 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247264ad-9fe8-490f-a6e9-8ea434e05aa8 None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:38.328515 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247264ad-9fe8-490f-a6e9-8ea434e05aa8 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:38.343821 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 414/819] 10.222.0.23 () {60 vars in 1144 bytes} [Thu May 19 11:58:38 2022] POST /identity/v3/endpoints => generated 346 bytes in 260 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 11:58:38.343821 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca6692e-2e71-444e-9c85-a7d40a443d43 None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:38.344471 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-247264ad-9fe8-490f-a6e9-8ea434e05aa8 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:38.344471 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-247264ad-9fe8-490f-a6e9-8ea434e05aa8 None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:38.361967 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application [None req-247264ad-9fe8-490f-a6e9-8ea434e05aa8 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 19 11:58:38.361967 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 11:58:38.361967 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 11:58:38.361967 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 11:58:38.361967 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 11:58:38.361967 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 11:58:38.361967 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 11:58:38.361967 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 11:58:38.361967 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 11:58:38.361967 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 11:58:38.361967 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 11:58:38.361967 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 11:58:38.361967 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 11:58:38.361967 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 11:58:38.361967 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 11:58:38.361967 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 11:58:38.363208 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 11:58:38.363208 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 11:58:38.363208 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 11:58:38.363208 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 11:58:38.363208 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 11:58:38.363208 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 11:58:38.363208 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 11:58:38.363208 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 11:58:38.363208 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 11:58:38.363208 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.get_or_create( May 19 11:58:38.363208 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 11:58:38.363208 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application with Lock( May 19 11:58:38.363208 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 11:58:38.363208 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._enter() May 19 11:58:38.363208 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 11:58:38.363208 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 11:58:38.363208 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 11:58:38.363208 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.creator() May 19 11:58:38.364312 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 11:58:38.364312 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application created_value = creator( May 19 11:58:38.364312 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 11:58:38.364312 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 11:58:38.364312 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 11:58:38.364312 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 11:58:38.364312 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 11:58:38.364312 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 11:58:38.364312 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 19 11:58:38.364312 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application May 19 11:58:38.364312 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 406/820] 10.222.0.23 () {56 vars in 1117 bytes} [Thu May 19 11:58:38 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 11:58:38.364929 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247264ad-9fe8-490f-a6e9-8ea434e05aa8 None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:38.393055 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:58:38.474491 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:58:38.484162 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247264ad-9fe8-490f-a6e9-8ea434e05aa8 None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:38.506293 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca6692e-2e71-444e-9c85-a7d40a443d43 None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:38.525975 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5e6507-a728-4013-ade0-2d3dc6f55f8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:38.526654 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5e6507-a728-4013-ade0-2d3dc6f55f8f None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:38.527000 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5e6507-a728-4013-ade0-2d3dc6f55f8f None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:38.527281 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5e6507-a728-4013-ade0-2d3dc6f55f8f None admin] PATH_INFO: `/v3/services` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:38.528037 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5e6507-a728-4013-ade0-2d3dc6f55f8f None admin] RBAC: Adding query filter params (name=network) {{(pid=80539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 11:58:38.528349 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5e6507-a728-4013-ade0-2d3dc6f55f8f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:38.530686 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5e6507-a728-4013-ade0-2d3dc6f55f8f None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:38.539445 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 407/821] 10.222.0.23 () {56 vars in 1112 bytes} [Thu May 19 11:58:38 2022] GET /identity/v3/services?name=network => generated 124 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:58:38.540534 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5e6507-a728-4013-ade0-2d3dc6f55f8f None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:38.555661 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f6ff17-9919-460a-bdca-beca678eb69c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:38.557133 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f6ff17-9919-460a-bdca-beca678eb69c None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:38.557721 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f6ff17-9919-460a-bdca-beca678eb69c None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:38.558345 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f6ff17-9919-460a-bdca-beca678eb69c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:38.568611 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f6ff17-9919-460a-bdca-beca678eb69c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:38.571632 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f6ff17-9919-460a-bdca-beca678eb69c None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:38.588741 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application [None req-97f6ff17-9919-460a-bdca-beca678eb69c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 11:58:38.588741 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 11:58:38.588741 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 11:58:38.588741 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 11:58:38.588741 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 11:58:38.588741 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 11:58:38.588741 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 11:58:38.588741 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 11:58:38.588741 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 11:58:38.588741 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 11:58:38.588741 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 11:58:38.588741 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 11:58:38.588741 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 11:58:38.588741 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 11:58:38.588741 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 11:58:38.589836 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 11:58:38.589836 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 11:58:38.589836 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 11:58:38.589836 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 11:58:38.589836 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 11:58:38.589836 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 11:58:38.589836 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.get_or_create( May 19 11:58:38.589836 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 11:58:38.589836 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application with Lock( May 19 11:58:38.589836 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 11:58:38.589836 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._enter() May 19 11:58:38.589836 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 11:58:38.589836 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 11:58:38.589836 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 11:58:38.589836 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.creator() May 19 11:58:38.589836 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 11:58:38.589836 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application created_value = creator( May 19 11:58:38.590922 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 11:58:38.590922 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 11:58:38.590922 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 11:58:38.590922 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 11:58:38.590922 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 11:58:38.590922 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 11:58:38.590922 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 11:58:38.590922 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application May 19 11:58:38.595637 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 415/822] 10.222.0.23 () {56 vars in 1111 bytes} [Thu May 19 11:58:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 11:58:38.596585 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f6ff17-9919-460a-bdca-beca678eb69c None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:38.640702 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:58:38.665276 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5e6507-a728-4013-ade0-2d3dc6f55f8f None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:38.703358 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147b5dba-af77-4c94-a2ae-46a00aa53f1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:38.704367 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147b5dba-af77-4c94-a2ae-46a00aa53f1b None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:38.704947 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147b5dba-af77-4c94-a2ae-46a00aa53f1b None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:38.705228 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147b5dba-af77-4c94-a2ae-46a00aa53f1b None admin] PATH_INFO: `/v3/users` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:38.706072 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147b5dba-af77-4c94-a2ae-46a00aa53f1b None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 11:58:38.706540 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147b5dba-af77-4c94-a2ae-46a00aa53f1b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:38.708703 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147b5dba-af77-4c94-a2ae-46a00aa53f1b None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:38.728647 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f6ff17-9919-460a-bdca-beca678eb69c None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:38.732713 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 408/823] 10.222.0.23 () {56 vars in 1149 bytes} [Thu May 19 11:58:38 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:58:38.733802 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147b5dba-af77-4c94-a2ae-46a00aa53f1b None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:38.772001 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7e2975-01bc-48d8-b7cc-e276dedb2f9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:38.772683 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7e2975-01bc-48d8-b7cc-e276dedb2f9d None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:38.773046 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7e2975-01bc-48d8-b7cc-e276dedb2f9d None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:38.773273 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7e2975-01bc-48d8-b7cc-e276dedb2f9d None admin] PATH_INFO: `/v3/services` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:38.774089 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7e2975-01bc-48d8-b7cc-e276dedb2f9d None admin] RBAC: Adding query filter params (type=network) {{(pid=80540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 11:58:38.774403 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7e2975-01bc-48d8-b7cc-e276dedb2f9d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:38.776062 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7e2975-01bc-48d8-b7cc-e276dedb2f9d None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:38.782463 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 416/824] 10.222.0.23 () {56 vars in 1112 bytes} [Thu May 19 11:58:38 2022] GET /identity/v3/services?type=network => generated 350 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:58:38.783507 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7e2975-01bc-48d8-b7cc-e276dedb2f9d None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:38.793458 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:58:38.846798 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147b5dba-af77-4c94-a2ae-46a00aa53f1b None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:38.883039 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8944ca9-5e04-4a64-bd2b-de3bafd173d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:38.883708 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8944ca9-5e04-4a64-bd2b-de3bafd173d4 None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:38.884095 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8944ca9-5e04-4a64-bd2b-de3bafd173d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:38.884386 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8944ca9-5e04-4a64-bd2b-de3bafd173d4 None admin] PATH_INFO: `/v3/projects` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:38.885184 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8944ca9-5e04-4a64-bd2b-de3bafd173d4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 11:58:38.885501 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8944ca9-5e04-4a64-bd2b-de3bafd173d4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:38.886936 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8944ca9-5e04-4a64-bd2b-de3bafd173d4 None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:38.894992 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:58:38.901696 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7e2975-01bc-48d8-b7cc-e276dedb2f9d None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:38.905628 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 409/825] 10.222.0.23 () {56 vars in 1148 bytes} [Thu May 19 11:58:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:58:38.906565 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8944ca9-5e04-4a64-bd2b-de3bafd173d4 None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:38.949181 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a30171-c845-4439-9a4d-f5f40861e7b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:38.950104 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a30171-c845-4439-9a4d-f5f40861e7b4 None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:38.950394 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a30171-c845-4439-9a4d-f5f40861e7b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:38.950671 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a30171-c845-4439-9a4d-f5f40861e7b4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:38.963823 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a30171-c845-4439-9a4d-f5f40861e7b4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:38.965329 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a30171-c845-4439-9a4d-f5f40861e7b4 None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:38.984940 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application [None req-49a30171-c845-4439-9a4d-f5f40861e7b4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 11:58:38.984940 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 11:58:38.984940 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 11:58:38.984940 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 11:58:38.984940 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 11:58:38.984940 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 11:58:38.984940 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 11:58:38.984940 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 11:58:38.984940 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 11:58:38.984940 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 11:58:38.984940 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 11:58:38.984940 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 11:58:38.984940 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 11:58:38.984940 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 11:58:38.984940 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 11:58:38.987002 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 11:58:38.987002 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 11:58:38.987002 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 11:58:38.987002 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 11:58:38.987002 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 11:58:38.987002 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 11:58:38.987002 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.get_or_create( May 19 11:58:38.987002 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 11:58:38.987002 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application with Lock( May 19 11:58:38.987002 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 11:58:38.987002 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._enter() May 19 11:58:38.987002 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 11:58:38.987002 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 11:58:38.987002 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 11:58:38.987002 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.creator() May 19 11:58:38.987002 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 11:58:38.987002 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application created_value = creator( May 19 11:58:38.989135 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 11:58:38.989135 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 11:58:38.989135 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 11:58:38.989135 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application May 19 11:58:38.989135 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 417/826] 10.222.0.23 () {56 vars in 1108 bytes} [Thu May 19 11:58:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 11:58:38.989135 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a30171-c845-4439-9a4d-f5f40861e7b4 None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:39.017549 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8944ca9-5e04-4a64-bd2b-de3bafd173d4 None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:39.056065 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:58:39.059700 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbfbcc1-dc11-4227-975e-e682ebd70e11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:39.061161 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbfbcc1-dc11-4227-975e-e682ebd70e11 None admin] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:39.061678 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbfbcc1-dc11-4227-975e-e682ebd70e11 None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:39.062461 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbfbcc1-dc11-4227-975e-e682ebd70e11 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:39.063692 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdbfbcc1-dc11-4227-975e-e682ebd70e11 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:39.065483 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdbfbcc1-dc11-4227-975e-e682ebd70e11 None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:39.103797 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a30171-c845-4439-9a4d-f5f40861e7b4 None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:39.141260 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98eb615-8682-410f-993e-d73d235c2511 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:39.141928 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98eb615-8682-410f-993e-d73d235c2511 None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:39.142214 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98eb615-8682-410f-993e-d73d235c2511 None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:39.142521 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98eb615-8682-410f-993e-d73d235c2511 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:39.143322 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98eb615-8682-410f-993e-d73d235c2511 None admin] RBAC: Adding query filter params (role.id=9db5671c65d3414ea2682afa2ccf1403, scope.project.id=d70d25bf69314820b8aaecb65576978a, user.id=34a98d75d84b4c67981377e0f26eca62) {{(pid=80540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 11:58:39.143638 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98eb615-8682-410f-993e-d73d235c2511 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:39.145216 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98eb615-8682-410f-993e-d73d235c2511 None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:39.151976 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 418/827] 10.222.0.23 () {56 vars in 1374 bytes} [Thu May 19 11:58:38 2022] GET /identity/v3/role_assignments?user.id=34a98d75d84b4c67981377e0f26eca62&scope.project.id=d70d25bf69314820b8aaecb65576978a&role.id=9db5671c65d3414ea2682afa2ccf1403 => generated 608 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:58:39.152543 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98eb615-8682-410f-993e-d73d235c2511 None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:39.193941 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 410/828] 10.222.0.23 () {60 vars in 1144 bytes} [Thu May 19 11:58:38 2022] POST /identity/v3/endpoints => generated 327 bytes in 288 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 11:58:39.282293 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98eb615-8682-410f-993e-d73d235c2511 None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:39.312613 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:58:39.335714 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02f29ca-4e67-4ce4-ba2b-a7a11314d739 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:39.336502 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02f29ca-4e67-4ce4-ba2b-a7a11314d739 None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:39.336788 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02f29ca-4e67-4ce4-ba2b-a7a11314d739 None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:39.337067 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02f29ca-4e67-4ce4-ba2b-a7a11314d739 None admin] PATH_INFO: `/v3/domains` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:39.337836 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02f29ca-4e67-4ce4-ba2b-a7a11314d739 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 11:58:39.338147 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02f29ca-4e67-4ce4-ba2b-a7a11314d739 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:39.339630 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02f29ca-4e67-4ce4-ba2b-a7a11314d739 None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:39.354464 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 419/829] 10.222.0.23 () {56 vars in 1109 bytes} [Thu May 19 11:58:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:58:39.360626 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbfbcc1-dc11-4227-975e-e682ebd70e11 None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:39.442400 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:58:39.496827 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbfbcc1-dc11-4227-975e-e682ebd70e11 None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:39.510729 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6703f191-7e11-472b-945d-e207dcb914a3 None None] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:39.511020 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6703f191-7e11-472b-945d-e207dcb914a3 None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:39.511301 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6703f191-7e11-472b-945d-e207dcb914a3 None None] PATH_INFO: `/` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:39.513170 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 420/830] 10.222.0.23 () {54 vars in 926 bytes} [Thu May 19 11:58:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 11:58:39.519991 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2285564-fa1e-4680-9699-6886a660a22a None None] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:39.520281 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2285564-fa1e-4680-9699-6886a660a22a None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:39.520732 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2285564-fa1e-4680-9699-6886a660a22a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:39.543401 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.auth.core [None req-c2285564-fa1e-4680-9699-6886a660a22a None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 11:58:39.545164 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-c2285564-fa1e-4680-9699-6886a660a22a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 11:58:39.547297 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94da1182-0a8e-408f-8f46-1e29bea9148e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:39.548747 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94da1182-0a8e-408f-8f46-1e29bea9148e None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:39.549193 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94da1182-0a8e-408f-8f46-1e29bea9148e None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:39.549473 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94da1182-0a8e-408f-8f46-1e29bea9148e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:39.556711 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94da1182-0a8e-408f-8f46-1e29bea9148e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:39.558185 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94da1182-0a8e-408f-8f46-1e29bea9148e None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:39.578765 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application [None req-94da1182-0a8e-408f-8f46-1e29bea9148e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 11:58:39.578765 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 11:58:39.578765 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 11:58:39.578765 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 11:58:39.578765 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 11:58:39.578765 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 11:58:39.578765 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 11:58:39.578765 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 11:58:39.578765 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 11:58:39.578765 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 11:58:39.578765 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 11:58:39.578765 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 11:58:39.578765 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 11:58:39.578765 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 11:58:39.578765 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 11:58:39.579729 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 11:58:39.579729 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 11:58:39.579729 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 11:58:39.579729 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 11:58:39.579729 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 11:58:39.579729 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 11:58:39.579729 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.get_or_create( May 19 11:58:39.579729 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 11:58:39.579729 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application with Lock( May 19 11:58:39.579729 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 11:58:39.579729 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._enter() May 19 11:58:39.579729 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 11:58:39.579729 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 11:58:39.579729 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 11:58:39.579729 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.creator() May 19 11:58:39.579729 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 11:58:39.579729 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application created_value = creator( May 19 11:58:39.580687 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 11:58:39.580687 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 11:58:39.580687 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 11:58:39.580687 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 11:58:39.580687 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 11:58:39.580687 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 11:58:39.580687 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 11:58:39.580687 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application May 19 11:58:39.580687 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 411/831] 10.222.0.23 () {56 vars in 1111 bytes} [Thu May 19 11:58:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 11:58:39.588819 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94da1182-0a8e-408f-8f46-1e29bea9148e None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:39.640544 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 421/832] 10.222.0.23 () {58 vars in 1023 bytes} [Thu May 19 11:58:39 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 11:58:39.667563 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2285564-fa1e-4680-9699-6886a660a22a None None] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:39.734342 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94da1182-0a8e-408f-8f46-1e29bea9148e None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:39.775820 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:58:39.780733 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d1778d-1a13-437c-953e-55d880bfbb87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:39.783667 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d1778d-1a13-437c-953e-55d880bfbb87 None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:39.783667 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d1778d-1a13-437c-953e-55d880bfbb87 None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:39.783667 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d1778d-1a13-437c-953e-55d880bfbb87 None admin] PATH_INFO: `/v3/projects` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:39.784408 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d1778d-1a13-437c-953e-55d880bfbb87 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 11:58:39.785126 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d1778d-1a13-437c-953e-55d880bfbb87 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:39.787504 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d1778d-1a13-437c-953e-55d880bfbb87 None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:39.792014 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2285564-fa1e-4680-9699-6886a660a22a None None] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:39.804291 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:58:39.811718 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 412/833] 10.222.0.23 () {56 vars in 1148 bytes} [Thu May 19 11:58:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:58:39.812654 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d1778d-1a13-437c-953e-55d880bfbb87 None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:39.848817 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af112548-34ba-477b-bfda-6108bb673c97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:39.849989 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af112548-34ba-477b-bfda-6108bb673c97 None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:39.850663 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af112548-34ba-477b-bfda-6108bb673c97 None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:39.851145 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af112548-34ba-477b-bfda-6108bb673c97 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:39.872605 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af112548-34ba-477b-bfda-6108bb673c97 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 11:58:39.884698 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application [None req-af112548-34ba-477b-bfda-6108bb673c97 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 11:58:39.884698 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 11:58:39.884698 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 11:58:39.884698 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 11:58:39.884698 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 11:58:39.884698 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 11:58:39.884698 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 11:58:39.884698 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 11:58:39.884698 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 11:58:39.884698 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 11:58:39.884698 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 11:58:39.884698 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 11:58:39.884698 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 11:58:39.884698 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 11:58:39.884698 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 11:58:39.884698 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 11:58:39.885869 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 11:58:39.885869 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 11:58:39.885869 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 11:58:39.885869 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 11:58:39.885869 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 11:58:39.885869 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 11:58:39.885869 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 11:58:39.885869 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 11:58:39.885869 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 11:58:39.885869 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.get_or_create( May 19 11:58:39.885869 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 11:58:39.885869 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application with Lock( May 19 11:58:39.885869 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 11:58:39.885869 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._enter() May 19 11:58:39.885869 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 11:58:39.885869 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 11:58:39.886889 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 11:58:39.886889 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.creator() May 19 11:58:39.886889 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 11:58:39.886889 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application created_value = creator( May 19 11:58:39.886889 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 11:58:39.886889 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 11:58:39.886889 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 11:58:39.886889 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 11:58:39.886889 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 11:58:39.886889 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 11:58:39.886889 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 11:58:39.886889 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application May 19 11:58:39.891025 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 422/834] 10.222.0.23 () {56 vars in 1096 bytes} [Thu May 19 11:58:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 11:58:39.899834 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af112548-34ba-477b-bfda-6108bb673c97 None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:39.939443 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d1778d-1a13-437c-953e-55d880bfbb87 None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:39.983195 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc081ed2-5ff4-4d86-85ca-4475193c5a44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:39.984146 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc081ed2-5ff4-4d86-85ca-4475193c5a44 None admin] REQUEST_METHOD: `PUT` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:39.984484 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc081ed2-5ff4-4d86-85ca-4475193c5a44 None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:39.984703 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc081ed2-5ff4-4d86-85ca-4475193c5a44 None admin] PATH_INFO: `/v3/projects/d70d25bf69314820b8aaecb65576978a/users/0c3181fb4f584a549a786adfefdf3dfb/roles/7c59061ba7064a0db6032433fb0eec6b` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:40.003237 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc081ed2-5ff4-4d86-85ca-4475193c5a44 None admin] RBAC: Authorizing `identity:create_grant(project_id=d70d25bf69314820b8aaecb65576978a, user_id=0c3181fb4f584a549a786adfefdf3dfb, role_id=7c59061ba7064a0db6032433fb0eec6b)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:40.004696 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc081ed2-5ff4-4d86-85ca-4475193c5a44 None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:40.040379 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af112548-34ba-477b-bfda-6108bb673c97 None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:40.075388 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 413/835] 10.222.0.23 () {58 vars in 1439 bytes} [Thu May 19 11:58:39 2022] PUT /identity/v3/projects/d70d25bf69314820b8aaecb65576978a/users/0c3181fb4f584a549a786adfefdf3dfb/roles/7c59061ba7064a0db6032433fb0eec6b => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 11:58:40.083844 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8143a87-ac2b-468e-9fc9-bb36c10c2970 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:40.083844 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8143a87-ac2b-468e-9fc9-bb36c10c2970 None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:40.083844 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8143a87-ac2b-468e-9fc9-bb36c10c2970 None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:40.083844 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8143a87-ac2b-468e-9fc9-bb36c10c2970 None admin] PATH_INFO: `/v3/roles` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:40.084285 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8143a87-ac2b-468e-9fc9-bb36c10c2970 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 11:58:40.084285 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8143a87-ac2b-468e-9fc9-bb36c10c2970 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:40.084285 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8143a87-ac2b-468e-9fc9-bb36c10c2970 None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:40.093900 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 423/836] 10.222.0.23 () {56 vars in 1099 bytes} [Thu May 19 11:58:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 11:58:40.099671 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc081ed2-5ff4-4d86-85ca-4475193c5a44 None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:40.133649 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:58:40.227951 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc081ed2-5ff4-4d86-85ca-4475193c5a44 None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:40.267540 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f46df1c-cb54-465d-bc54-b79b03cc13d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:40.268283 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f46df1c-cb54-465d-bc54-b79b03cc13d1 None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:40.268568 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f46df1c-cb54-465d-bc54-b79b03cc13d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:40.268838 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f46df1c-cb54-465d-bc54-b79b03cc13d1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:40.284002 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f46df1c-cb54-465d-bc54-b79b03cc13d1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:40.285488 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f46df1c-cb54-465d-bc54-b79b03cc13d1 None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:40.304529 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application [None req-3f46df1c-cb54-465d-bc54-b79b03cc13d1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 11:58:40.304529 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 11:58:40.304529 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 11:58:40.304529 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 11:58:40.304529 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 11:58:40.304529 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 11:58:40.304529 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 11:58:40.304529 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 11:58:40.304529 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 11:58:40.304529 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 11:58:40.304529 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 11:58:40.304529 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 11:58:40.304529 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 11:58:40.304529 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 11:58:40.304529 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 11:58:40.305464 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 11:58:40.305464 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 11:58:40.305464 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 11:58:40.305464 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 11:58:40.305464 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 11:58:40.305464 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 11:58:40.305464 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.get_or_create( May 19 11:58:40.305464 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 11:58:40.305464 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application with Lock( May 19 11:58:40.305464 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 11:58:40.305464 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._enter() May 19 11:58:40.305464 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 11:58:40.305464 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 11:58:40.305464 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 11:58:40.305464 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.creator() May 19 11:58:40.305464 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 11:58:40.305464 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application created_value = creator( May 19 11:58:40.306512 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 11:58:40.306512 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 11:58:40.306512 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 11:58:40.306512 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application May 19 11:58:40.306512 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 414/837] 10.222.0.23 () {56 vars in 1108 bytes} [Thu May 19 11:58:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 11:58:40.311387 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8143a87-ac2b-468e-9fc9-bb36c10c2970 None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:40.350374 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:58:40.427530 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8143a87-ac2b-468e-9fc9-bb36c10c2970 None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:40.464595 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0fed55-7242-40d5-9550-ff3a1db2f370 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:40.465272 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0fed55-7242-40d5-9550-ff3a1db2f370 None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:40.465562 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0fed55-7242-40d5-9550-ff3a1db2f370 None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:40.465845 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0fed55-7242-40d5-9550-ff3a1db2f370 None admin] PATH_INFO: `/v3/domains` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:40.467101 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0fed55-7242-40d5-9550-ff3a1db2f370 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 11:58:40.467418 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0fed55-7242-40d5-9550-ff3a1db2f370 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:40.468837 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0fed55-7242-40d5-9550-ff3a1db2f370 None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:40.481808 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 424/838] 10.222.0.23 () {56 vars in 1109 bytes} [Thu May 19 11:58:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:58:40.487762 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f46df1c-cb54-465d-bc54-b79b03cc13d1 None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:40.579636 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:58:40.603465 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f46df1c-cb54-465d-bc54-b79b03cc13d1 None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:40.641617 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85111bb5-1c5b-40c7-9759-b7da558c1cb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:40.642419 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85111bb5-1c5b-40c7-9759-b7da558c1cb7 None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:40.642750 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85111bb5-1c5b-40c7-9759-b7da558c1cb7 None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:40.643035 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85111bb5-1c5b-40c7-9759-b7da558c1cb7 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:40.650831 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85111bb5-1c5b-40c7-9759-b7da558c1cb7 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:40.652388 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85111bb5-1c5b-40c7-9759-b7da558c1cb7 None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:40.667103 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application [None req-85111bb5-1c5b-40c7-9759-b7da558c1cb7 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 19 11:58:40.667103 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 11:58:40.667103 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 11:58:40.667103 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 11:58:40.667103 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 11:58:40.667103 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 11:58:40.667103 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 11:58:40.667103 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 11:58:40.667103 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 11:58:40.667103 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 11:58:40.667103 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 11:58:40.667103 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 11:58:40.667103 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 11:58:40.667103 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 11:58:40.667103 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 11:58:40.667103 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 11:58:40.668105 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 11:58:40.668105 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 11:58:40.668105 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 11:58:40.668105 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 11:58:40.668105 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 11:58:40.668105 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 11:58:40.668105 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 11:58:40.668105 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 11:58:40.668105 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 11:58:40.668105 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.get_or_create( May 19 11:58:40.668105 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 11:58:40.668105 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application with Lock( May 19 11:58:40.668105 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 11:58:40.668105 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._enter() May 19 11:58:40.668105 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 11:58:40.668105 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 11:58:40.668105 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 11:58:40.668105 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.creator() May 19 11:58:40.669015 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 11:58:40.669015 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application created_value = creator( May 19 11:58:40.669015 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 11:58:40.669015 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 11:58:40.669015 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 11:58:40.669015 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 11:58:40.669015 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 11:58:40.669015 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 11:58:40.669015 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 19 11:58:40.669015 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application May 19 11:58:40.669015 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 415/839] 10.222.0.23 () {56 vars in 1096 bytes} [Thu May 19 11:58:40 2022] GET /identity/v3/users/swift => generated 83 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 11:58:40.674211 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0fed55-7242-40d5-9550-ff3a1db2f370 None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:40.781242 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:58:40.792527 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0fed55-7242-40d5-9550-ff3a1db2f370 None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:40.834189 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1121773-ebbf-43cf-915c-749b262b4638 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:40.834783 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1121773-ebbf-43cf-915c-749b262b4638 None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:40.835072 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1121773-ebbf-43cf-915c-749b262b4638 None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:40.835355 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1121773-ebbf-43cf-915c-749b262b4638 None admin] PATH_INFO: `/v3/users` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:40.836224 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1121773-ebbf-43cf-915c-749b262b4638 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 11:58:40.836541 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1121773-ebbf-43cf-915c-749b262b4638 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:40.838023 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1121773-ebbf-43cf-915c-749b262b4638 None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:40.858429 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 425/840] 10.222.0.23 () {56 vars in 1135 bytes} [Thu May 19 11:58:40 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:58:40.863570 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85111bb5-1c5b-40c7-9759-b7da558c1cb7 None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:40.970248 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85111bb5-1c5b-40c7-9759-b7da558c1cb7 None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:40.989050 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27dca4da-bef5-495e-9217-56b3d6bf5fb8 None None] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:40.989375 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27dca4da-bef5-495e-9217-56b3d6bf5fb8 None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:40.989615 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27dca4da-bef5-495e-9217-56b3d6bf5fb8 None None] PATH_INFO: `/` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:40.991231 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 426/841] 10.222.0.23 () {54 vars in 926 bytes} [Thu May 19 11:58:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 11:58:40.996396 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839bd5dd-0fa6-4bef-97d6-d572eebf553d None None] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:40.996689 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839bd5dd-0fa6-4bef-97d6-d572eebf553d None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:40.996970 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839bd5dd-0fa6-4bef-97d6-d572eebf553d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:41.009552 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261e5d07-867f-4c2e-81af-38ef6ab59408 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:41.010218 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261e5d07-867f-4c2e-81af-38ef6ab59408 None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:41.010513 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261e5d07-867f-4c2e-81af-38ef6ab59408 None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:41.010791 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261e5d07-867f-4c2e-81af-38ef6ab59408 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:41.016023 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.auth.core [None req-839bd5dd-0fa6-4bef-97d6-d572eebf553d None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 11:58:41.016235 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:58:41.017674 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-839bd5dd-0fa6-4bef-97d6-d572eebf553d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 11:58:41.087918 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261e5d07-867f-4c2e-81af-38ef6ab59408 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:41.089928 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261e5d07-867f-4c2e-81af-38ef6ab59408 None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:41.091204 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 427/842] 10.222.0.23 () {58 vars in 1023 bytes} [Thu May 19 11:58:40 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 11:58:41.113481 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839bd5dd-0fa6-4bef-97d6-d572eebf553d None None] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:41.200921 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:58:41.206584 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 416/843] 10.222.0.23 () {58 vars in 1282 bytes} [Thu May 19 11:58:40 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 11:58:41.213145 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261e5d07-867f-4c2e-81af-38ef6ab59408 None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:41.228808 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839bd5dd-0fa6-4bef-97d6-d572eebf553d None None] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:41.254720 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:58:41.272823 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f5bcad-4f3c-43bb-9a91-f223a7ca31f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:41.282542 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f5bcad-4f3c-43bb-9a91-f223a7ca31f5 None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:41.282542 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f5bcad-4f3c-43bb-9a91-f223a7ca31f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:41.282542 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f5bcad-4f3c-43bb-9a91-f223a7ca31f5 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:41.282542 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f5bcad-4f3c-43bb-9a91-f223a7ca31f5 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:41.282542 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f5bcad-4f3c-43bb-9a91-f223a7ca31f5 None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:41.295475 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application [None req-80f5bcad-4f3c-43bb-9a91-f223a7ca31f5 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. May 19 11:58:41.295475 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 11:58:41.295475 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 11:58:41.295475 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 11:58:41.295475 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 11:58:41.295475 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 11:58:41.295475 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 11:58:41.295475 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 11:58:41.295475 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 19 11:58:41.295475 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application May 19 11:58:41.295475 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 11:58:41.295475 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application May 19 11:58:41.295475 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 11:58:41.295475 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 11:58:41.295475 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 11:58:41.295475 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 11:58:41.295475 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 11:58:41.296544 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 11:58:41.296544 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 11:58:41.296544 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 11:58:41.296544 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 11:58:41.296544 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 11:58:41.296544 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 11:58:41.296544 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 11:58:41.296544 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 11:58:41.296544 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 11:58:41.296544 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 11:58:41.296544 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 11:58:41.296544 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 11:58:41.296544 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 11:58:41.296544 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 11:58:41.296544 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 11:58:41.296544 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.get_or_create( May 19 11:58:41.297398 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 11:58:41.297398 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application with Lock( May 19 11:58:41.297398 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 11:58:41.297398 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._enter() May 19 11:58:41.297398 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 11:58:41.297398 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 11:58:41.297398 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 11:58:41.297398 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.creator() May 19 11:58:41.297398 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 11:58:41.297398 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application created_value = creator( May 19 11:58:41.297398 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 11:58:41.297398 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 11:58:41.297398 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 19 11:58:41.297398 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application May 19 11:58:41.300899 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 428/844] 10.222.0.23 () {56 vars in 1114 bytes} [Thu May 19 11:58:41 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 11:58:41.306476 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f5bcad-4f3c-43bb-9a91-f223a7ca31f5 None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:41.337149 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261e5d07-867f-4c2e-81af-38ef6ab59408 None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:41.385958 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dde14b6-6ae8-4d96-b01c-8758c877ca29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:41.387093 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dde14b6-6ae8-4d96-b01c-8758c877ca29 None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:41.387598 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dde14b6-6ae8-4d96-b01c-8758c877ca29 None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:41.388721 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dde14b6-6ae8-4d96-b01c-8758c877ca29 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:41.403810 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dde14b6-6ae8-4d96-b01c-8758c877ca29 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:41.406220 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dde14b6-6ae8-4d96-b01c-8758c877ca29 None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:41.414612 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f5bcad-4f3c-43bb-9a91-f223a7ca31f5 None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:41.428639 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application [None req-3dde14b6-6ae8-4d96-b01c-8758c877ca29 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 11:58:41.428639 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 11:58:41.428639 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 11:58:41.428639 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 11:58:41.428639 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 11:58:41.428639 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 11:58:41.428639 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 11:58:41.428639 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 11:58:41.428639 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 11:58:41.428639 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 11:58:41.428639 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 11:58:41.428639 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 11:58:41.428639 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 11:58:41.428639 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 11:58:41.428639 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 11:58:41.429565 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 11:58:41.429565 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 11:58:41.429565 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 11:58:41.429565 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 11:58:41.429565 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 11:58:41.429565 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 11:58:41.429565 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.get_or_create( May 19 11:58:41.429565 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 11:58:41.429565 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application with Lock( May 19 11:58:41.429565 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 11:58:41.429565 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._enter() May 19 11:58:41.429565 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 11:58:41.429565 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 11:58:41.429565 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 11:58:41.429565 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.creator() May 19 11:58:41.429565 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 11:58:41.429565 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application created_value = creator( May 19 11:58:41.430438 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 11:58:41.430438 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 11:58:41.430438 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 11:58:41.430438 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application May 19 11:58:41.431482 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 417/845] 10.222.0.23 () {56 vars in 1108 bytes} [Thu May 19 11:58:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 11:58:41.437830 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dde14b6-6ae8-4d96-b01c-8758c877ca29 None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:41.463393 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcef67f6-a75c-4c6c-b55c-98466ffcbea6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:41.464881 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcef67f6-a75c-4c6c-b55c-98466ffcbea6 None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:41.465384 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcef67f6-a75c-4c6c-b55c-98466ffcbea6 None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:41.465909 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcef67f6-a75c-4c6c-b55c-98466ffcbea6 None admin] PATH_INFO: `/v3/services` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:41.466835 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcef67f6-a75c-4c6c-b55c-98466ffcbea6 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 11:58:41.467354 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcef67f6-a75c-4c6c-b55c-98466ffcbea6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:41.469176 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcef67f6-a75c-4c6c-b55c-98466ffcbea6 None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:41.474743 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 429/846] 10.222.0.23 () {56 vars in 1114 bytes} [Thu May 19 11:58:41 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 11:58:41.479866 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcef67f6-a75c-4c6c-b55c-98466ffcbea6 None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:41.508371 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:58:41.556897 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:58:41.571819 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dde14b6-6ae8-4d96-b01c-8758c877ca29 None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:41.605770 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcef67f6-a75c-4c6c-b55c-98466ffcbea6 None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:41.615003 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3f74f1-96b0-4618-93c6-52eefa3573fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:41.615828 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3f74f1-96b0-4618-93c6-52eefa3573fa None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:41.616214 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3f74f1-96b0-4618-93c6-52eefa3573fa None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:41.616593 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3f74f1-96b0-4618-93c6-52eefa3573fa None admin] PATH_INFO: `/v3/domains` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:41.617979 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3f74f1-96b0-4618-93c6-52eefa3573fa None admin] RBAC: Adding query filter params (name=Default) {{(pid=80539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 11:58:41.618454 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3f74f1-96b0-4618-93c6-52eefa3573fa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:41.620406 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3f74f1-96b0-4618-93c6-52eefa3573fa None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:41.636427 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 418/847] 10.222.0.23 () {56 vars in 1109 bytes} [Thu May 19 11:58:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:58:41.642349 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3f74f1-96b0-4618-93c6-52eefa3573fa None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:41.652355 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2018b28-4735-4db3-8244-fb4f81d00092 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:41.653280 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2018b28-4735-4db3-8244-fb4f81d00092 None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:41.653800 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2018b28-4735-4db3-8244-fb4f81d00092 None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:41.654287 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2018b28-4735-4db3-8244-fb4f81d00092 None admin] PATH_INFO: `/v3/services` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:41.655349 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2018b28-4735-4db3-8244-fb4f81d00092 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 11:58:41.655868 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2018b28-4735-4db3-8244-fb4f81d00092 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:41.657530 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2018b28-4735-4db3-8244-fb4f81d00092 None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:41.671938 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 430/848] 10.222.0.23 () {56 vars in 1114 bytes} [Thu May 19 11:58:41 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:58:41.706265 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac4eab3-96f8-488c-a954-ccddfc199fc9 None None] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:41.706587 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac4eab3-96f8-488c-a954-ccddfc199fc9 None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:41.706830 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac4eab3-96f8-488c-a954-ccddfc199fc9 None None] PATH_INFO: `/` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:41.708502 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 431/849] 10.222.0.23 () {54 vars in 926 bytes} [Thu May 19 11:58:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 11:58:41.714800 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d335c05-8823-4d86-adbf-6cb2d867453d None None] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:41.715342 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d335c05-8823-4d86-adbf-6cb2d867453d None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:41.715768 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d335c05-8823-4d86-adbf-6cb2d867453d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:41.738514 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.auth.core [None req-8d335c05-8823-4d86-adbf-6cb2d867453d None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 11:58:41.740171 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-8d335c05-8823-4d86-adbf-6cb2d867453d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 11:58:41.774136 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3f74f1-96b0-4618-93c6-52eefa3573fa None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:41.822331 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7b7c7c-0a43-4039-9638-7bce7dfc90cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:41.823435 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7b7c7c-0a43-4039-9638-7bce7dfc90cd None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:41.823722 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7b7c7c-0a43-4039-9638-7bce7dfc90cd None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:41.824032 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7b7c7c-0a43-4039-9638-7bce7dfc90cd None admin] PATH_INFO: `/v3/projects/service` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:41.827262 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:58:41.837728 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7b7c7c-0a43-4039-9638-7bce7dfc90cd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:41.837728 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7b7c7c-0a43-4039-9638-7bce7dfc90cd None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:41.838450 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 432/850] 10.222.0.23 () {58 vars in 1023 bytes} [Thu May 19 11:58:41 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 11:58:41.852295 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application [None req-da7b7c7c-0a43-4039-9638-7bce7dfc90cd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 11:58:41.852295 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 11:58:41.852295 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 11:58:41.852295 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 11:58:41.852295 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 11:58:41.852295 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 11:58:41.852295 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 11:58:41.852295 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 11:58:41.852295 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 11:58:41.852295 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 11:58:41.852295 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 11:58:41.852295 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 11:58:41.852295 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 11:58:41.852295 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 11:58:41.852295 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 11:58:41.853269 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 11:58:41.853269 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 11:58:41.853269 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 11:58:41.853269 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 11:58:41.853269 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 11:58:41.853269 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 11:58:41.853269 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.get_or_create( May 19 11:58:41.853269 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 11:58:41.853269 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application with Lock( May 19 11:58:41.853269 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 11:58:41.853269 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._enter() May 19 11:58:41.853269 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 11:58:41.853269 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 11:58:41.853269 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 11:58:41.853269 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.creator() May 19 11:58:41.853269 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 11:58:41.853269 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application created_value = creator( May 19 11:58:41.854237 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 11:58:41.854237 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 11:58:41.854237 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 11:58:41.854237 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 11:58:41.854237 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 11:58:41.854237 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 11:58:41.854237 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 11:58:41.854237 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application May 19 11:58:41.858875 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 419/851] 10.222.0.23 () {56 vars in 1111 bytes} [Thu May 19 11:58:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 11:58:41.860347 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d335c05-8823-4d86-adbf-6cb2d867453d None None] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:41.862452 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7b7c7c-0a43-4039-9638-7bce7dfc90cd None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:41.876244 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:58:41.983124 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d335c05-8823-4d86-adbf-6cb2d867453d None None] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:41.997609 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7b7c7c-0a43-4039-9638-7bce7dfc90cd None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:42.033415 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f591857-173c-48fc-b1aa-e4d8f3b37c15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:42.034116 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f591857-173c-48fc-b1aa-e4d8f3b37c15 None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:42.034372 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f591857-173c-48fc-b1aa-e4d8f3b37c15 None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:42.034651 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f591857-173c-48fc-b1aa-e4d8f3b37c15 None admin] PATH_INFO: `/v3/projects` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:42.035460 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f591857-173c-48fc-b1aa-e4d8f3b37c15 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 11:58:42.035776 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f591857-173c-48fc-b1aa-e4d8f3b37c15 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:42.037261 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f591857-173c-48fc-b1aa-e4d8f3b37c15 None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:42.037838 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf41da9e-cd32-4e40-bb61-e6135b4a788e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:42.038888 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf41da9e-cd32-4e40-bb61-e6135b4a788e None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:42.039350 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf41da9e-cd32-4e40-bb61-e6135b4a788e None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:42.039833 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf41da9e-cd32-4e40-bb61-e6135b4a788e None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:42.040666 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf41da9e-cd32-4e40-bb61-e6135b4a788e None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:42.042207 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf41da9e-cd32-4e40-bb61-e6135b4a788e None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:42.056764 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 433/852] 10.222.0.23 () {56 vars in 1148 bytes} [Thu May 19 11:58:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:58:42.058446 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f591857-173c-48fc-b1aa-e4d8f3b37c15 None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:42.059529 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application [None req-cf41da9e-cd32-4e40-bb61-e6135b4a788e None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 19 11:58:42.059529 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 11:58:42.059529 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 11:58:42.059529 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 11:58:42.059529 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 11:58:42.059529 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 11:58:42.059529 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 11:58:42.059529 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 11:58:42.059529 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 19 11:58:42.059529 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application May 19 11:58:42.059529 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 11:58:42.059529 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application May 19 11:58:42.059529 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 11:58:42.059529 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 11:58:42.059529 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 11:58:42.059529 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 11:58:42.061113 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 11:58:42.061113 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 11:58:42.061113 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 11:58:42.061113 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 11:58:42.061113 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 11:58:42.061113 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 11:58:42.061113 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 11:58:42.061113 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 11:58:42.061113 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 11:58:42.061113 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 11:58:42.061113 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 11:58:42.061113 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 11:58:42.061113 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 11:58:42.061113 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 11:58:42.061113 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 11:58:42.061113 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 11:58:42.062064 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.get_or_create( May 19 11:58:42.062064 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 11:58:42.062064 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application with Lock( May 19 11:58:42.062064 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 11:58:42.062064 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._enter() May 19 11:58:42.062064 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 11:58:42.062064 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 11:58:42.062064 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 11:58:42.062064 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.creator() May 19 11:58:42.062064 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 11:58:42.062064 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application created_value = creator( May 19 11:58:42.062064 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 11:58:42.062064 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 11:58:42.062064 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 19 11:58:42.062064 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application May 19 11:58:42.063607 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 420/853] 10.222.0.23 () {56 vars in 1132 bytes} [Thu May 19 11:58:41 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 11:58:42.068312 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf41da9e-cd32-4e40-bb61-e6135b4a788e None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:42.133226 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:58:42.155449 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:58:42.188986 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf41da9e-cd32-4e40-bb61-e6135b4a788e None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:42.194501 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f591857-173c-48fc-b1aa-e4d8f3b37c15 None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:42.229730 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411219e9-248b-482d-ad7f-d8fb7b46931d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:42.230445 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411219e9-248b-482d-ad7f-d8fb7b46931d None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:42.230801 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411219e9-248b-482d-ad7f-d8fb7b46931d None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:42.231155 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411219e9-248b-482d-ad7f-d8fb7b46931d None admin] PATH_INFO: `/v3/services` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:42.231993 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411219e9-248b-482d-ad7f-d8fb7b46931d None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 11:58:42.232970 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411219e9-248b-482d-ad7f-d8fb7b46931d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:42.234606 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411219e9-248b-482d-ad7f-d8fb7b46931d None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:42.243835 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 421/854] 10.222.0.23 () {56 vars in 1126 bytes} [Thu May 19 11:58:42 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:58:42.246470 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411219e9-248b-482d-ad7f-d8fb7b46931d None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:42.252703 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc855b59-e79f-4a30-93d6-b4dbfd5ea8b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:42.253225 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc855b59-e79f-4a30-93d6-b4dbfd5ea8b7 None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:42.253511 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc855b59-e79f-4a30-93d6-b4dbfd5ea8b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:42.253793 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc855b59-e79f-4a30-93d6-b4dbfd5ea8b7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:42.254633 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc855b59-e79f-4a30-93d6-b4dbfd5ea8b7 None admin] RBAC: Adding query filter params (role.id=9db5671c65d3414ea2682afa2ccf1403, scope.project.id=d70d25bf69314820b8aaecb65576978a, user.id=15cbfc73ae1243d2a8b1e70f9cd99876) {{(pid=80540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 11:58:42.254953 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc855b59-e79f-4a30-93d6-b4dbfd5ea8b7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:42.256447 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc855b59-e79f-4a30-93d6-b4dbfd5ea8b7 None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:42.267943 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 434/855] 10.222.0.23 () {56 vars in 1374 bytes} [Thu May 19 11:58:42 2022] GET /identity/v3/role_assignments?user.id=15cbfc73ae1243d2a8b1e70f9cd99876&scope.project.id=d70d25bf69314820b8aaecb65576978a&role.id=9db5671c65d3414ea2682afa2ccf1403 => generated 608 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 11:58:42.389206 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411219e9-248b-482d-ad7f-d8fb7b46931d None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:42.438679 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29910d04-4ff1-415c-87b5-6c12a5c6c589 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:42.439822 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29910d04-4ff1-415c-87b5-6c12a5c6c589 None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:42.440350 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29910d04-4ff1-415c-87b5-6c12a5c6c589 None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:42.440843 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29910d04-4ff1-415c-87b5-6c12a5c6c589 None admin] PATH_INFO: `/v3/services` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:42.441793 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29910d04-4ff1-415c-87b5-6c12a5c6c589 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 11:58:42.442347 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29910d04-4ff1-415c-87b5-6c12a5c6c589 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:42.444135 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29910d04-4ff1-415c-87b5-6c12a5c6c589 None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:42.447254 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:58:42.450914 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 422/856] 10.222.0.23 () {56 vars in 1126 bytes} [Thu May 19 11:58:42 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 11:58:42.526943 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b980032e-3cd5-4b98-8ef3-f3dd23e8b49f None None] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:42.527565 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b980032e-3cd5-4b98-8ef3-f3dd23e8b49f None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:42.527565 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b980032e-3cd5-4b98-8ef3-f3dd23e8b49f None None] PATH_INFO: `/` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:42.529727 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 435/857] 10.222.0.23 () {54 vars in 926 bytes} [Thu May 19 11:58:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 11:58:42.536036 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973b02fc-0e2a-4e1e-b4c9-4a739e5e5ac5 None None] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:42.536329 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973b02fc-0e2a-4e1e-b4c9-4a739e5e5ac5 None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:42.536606 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973b02fc-0e2a-4e1e-b4c9-4a739e5e5ac5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:42.557354 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.auth.core [None req-973b02fc-0e2a-4e1e-b4c9-4a739e5e5ac5 None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 11:58:42.558592 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-973b02fc-0e2a-4e1e-b4c9-4a739e5e5ac5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 11:58:42.649513 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 423/858] 10.222.0.23 () {58 vars in 1023 bytes} [Thu May 19 11:58:42 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 11:58:42.674309 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b980032e-3cd5-4b98-8ef3-f3dd23e8b49f None None] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:42.789722 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b980032e-3cd5-4b98-8ef3-f3dd23e8b49f None None] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:42.822678 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:58:42.832846 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ea2456-1e6e-4891-a249-4c435f065668 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:42.833710 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ea2456-1e6e-4891-a249-4c435f065668 None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:42.833996 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ea2456-1e6e-4891-a249-4c435f065668 None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:42.834274 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ea2456-1e6e-4891-a249-4c435f065668 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:42.846971 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ea2456-1e6e-4891-a249-4c435f065668 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 11:58:42.855176 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application [None req-f8ea2456-1e6e-4891-a249-4c435f065668 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 11:58:42.855176 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 11:58:42.855176 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 11:58:42.855176 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 11:58:42.855176 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 11:58:42.855176 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 11:58:42.855176 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 11:58:42.855176 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 11:58:42.855176 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 11:58:42.855176 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 11:58:42.855176 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 11:58:42.855176 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 11:58:42.855176 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 11:58:42.855176 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 11:58:42.855176 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 11:58:42.855176 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 11:58:42.856364 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 11:58:42.856364 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 11:58:42.856364 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 11:58:42.856364 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 11:58:42.856364 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 11:58:42.856364 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 11:58:42.856364 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 11:58:42.856364 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 11:58:42.856364 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 11:58:42.856364 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.get_or_create( May 19 11:58:42.856364 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 11:58:42.856364 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application with Lock( May 19 11:58:42.856364 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 11:58:42.856364 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._enter() May 19 11:58:42.856364 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 11:58:42.856364 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 11:58:42.857383 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 11:58:42.857383 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.creator() May 19 11:58:42.857383 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 11:58:42.857383 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application created_value = creator( May 19 11:58:42.857383 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 11:58:42.857383 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 11:58:42.857383 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 11:58:42.857383 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 11:58:42.857383 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 11:58:42.857383 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 11:58:42.857383 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 11:58:42.857383 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application May 19 11:58:42.860063 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 436/859] 10.222.0.23 () {56 vars in 1102 bytes} [Thu May 19 11:58:42 2022] GET /identity/v3/roles/service => generated 85 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 11:58:42.865704 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973b02fc-0e2a-4e1e-b4c9-4a739e5e5ac5 None None] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:42.994229 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973b02fc-0e2a-4e1e-b4c9-4a739e5e5ac5 None None] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:43.028156 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:58:43.035588 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3135f20-6af7-4570-8b3f-e6f0b230bea4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:43.040025 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3135f20-6af7-4570-8b3f-e6f0b230bea4 None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:43.040025 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3135f20-6af7-4570-8b3f-e6f0b230bea4 None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:43.040025 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3135f20-6af7-4570-8b3f-e6f0b230bea4 None admin] PATH_INFO: `/v3/roles` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:43.040025 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3135f20-6af7-4570-8b3f-e6f0b230bea4 None admin] RBAC: Adding query filter params (name=service) {{(pid=80539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 11:58:43.040025 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3135f20-6af7-4570-8b3f-e6f0b230bea4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:43.040025 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3135f20-6af7-4570-8b3f-e6f0b230bea4 None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:43.050202 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 424/860] 10.222.0.23 () {56 vars in 1103 bytes} [Thu May 19 11:58:42 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:58:43.055968 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ea2456-1e6e-4891-a249-4c435f065668 None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:43.182480 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ea2456-1e6e-4891-a249-4c435f065668 None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:43.221734 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b15d29-6fb0-4cbe-a0d8-7d568db356fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:43.222614 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b15d29-6fb0-4cbe-a0d8-7d568db356fa None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:43.222904 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b15d29-6fb0-4cbe-a0d8-7d568db356fa None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:43.223187 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b15d29-6fb0-4cbe-a0d8-7d568db356fa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:43.238040 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b15d29-6fb0-4cbe-a0d8-7d568db356fa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:43.239543 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b15d29-6fb0-4cbe-a0d8-7d568db356fa None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:43.258998 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application [None req-b1b15d29-6fb0-4cbe-a0d8-7d568db356fa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 11:58:43.258998 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 11:58:43.258998 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 11:58:43.258998 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 11:58:43.258998 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 11:58:43.258998 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 11:58:43.258998 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 11:58:43.258998 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 11:58:43.258998 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 11:58:43.258998 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 11:58:43.258998 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 11:58:43.258998 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 11:58:43.258998 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 11:58:43.258998 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 11:58:43.258998 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 11:58:43.260002 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 11:58:43.260002 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 11:58:43.260002 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 11:58:43.260002 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 11:58:43.260002 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 11:58:43.260002 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 11:58:43.260002 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.get_or_create( May 19 11:58:43.260002 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 11:58:43.260002 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application with Lock( May 19 11:58:43.260002 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 11:58:43.260002 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._enter() May 19 11:58:43.260002 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 11:58:43.260002 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 11:58:43.260002 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 11:58:43.260002 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.creator() May 19 11:58:43.260002 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 11:58:43.260002 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application created_value = creator( May 19 11:58:43.261334 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 11:58:43.261334 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 11:58:43.261334 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 11:58:43.261334 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application May 19 11:58:43.261334 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 437/861] 10.222.0.23 () {56 vars in 1108 bytes} [Thu May 19 11:58:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 11:58:43.267368 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3135f20-6af7-4570-8b3f-e6f0b230bea4 None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:43.381866 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3135f20-6af7-4570-8b3f-e6f0b230bea4 None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:43.418071 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519316f1-dfd0-4a36-9544-1760613d2814 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:43.418744 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519316f1-dfd0-4a36-9544-1760613d2814 None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:43.419031 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519316f1-dfd0-4a36-9544-1760613d2814 None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:43.419310 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519316f1-dfd0-4a36-9544-1760613d2814 None admin] PATH_INFO: `/v3/domains` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:43.420115 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519316f1-dfd0-4a36-9544-1760613d2814 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 11:58:43.420461 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519316f1-dfd0-4a36-9544-1760613d2814 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:43.421809 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519316f1-dfd0-4a36-9544-1760613d2814 None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:43.435065 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 425/862] 10.222.0.23 () {56 vars in 1109 bytes} [Thu May 19 11:58:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:58:43.442465 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b15d29-6fb0-4cbe-a0d8-7d568db356fa None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:43.484161 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:58:43.567458 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b15d29-6fb0-4cbe-a0d8-7d568db356fa None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:43.614122 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff13267-d3c0-4fcb-9766-fa6dca7da0b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:43.614122 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff13267-d3c0-4fcb-9766-fa6dca7da0b4 None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:43.614122 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff13267-d3c0-4fcb-9766-fa6dca7da0b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:43.614122 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff13267-d3c0-4fcb-9766-fa6dca7da0b4 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:43.622674 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff13267-d3c0-4fcb-9766-fa6dca7da0b4 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:43.622674 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff13267-d3c0-4fcb-9766-fa6dca7da0b4 None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:43.638734 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application [None req-bff13267-d3c0-4fcb-9766-fa6dca7da0b4 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 19 11:58:43.638734 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 11:58:43.638734 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 11:58:43.638734 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 11:58:43.638734 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 11:58:43.638734 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 11:58:43.638734 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 11:58:43.638734 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 11:58:43.638734 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 11:58:43.638734 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 11:58:43.638734 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 11:58:43.638734 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 11:58:43.638734 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 11:58:43.638734 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 11:58:43.638734 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 11:58:43.638734 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 11:58:43.639854 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 11:58:43.639854 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 11:58:43.639854 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 11:58:43.639854 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 11:58:43.639854 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 11:58:43.639854 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 11:58:43.639854 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 11:58:43.639854 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 11:58:43.639854 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 11:58:43.639854 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.get_or_create( May 19 11:58:43.639854 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 11:58:43.639854 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application with Lock( May 19 11:58:43.639854 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 11:58:43.639854 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._enter() May 19 11:58:43.639854 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 11:58:43.639854 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 11:58:43.639854 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 11:58:43.639854 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.creator() May 19 11:58:43.640805 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 11:58:43.640805 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application created_value = creator( May 19 11:58:43.640805 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 11:58:43.640805 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 11:58:43.640805 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 11:58:43.640805 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 11:58:43.640805 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 11:58:43.640805 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 11:58:43.640805 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 19 11:58:43.640805 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application May 19 11:58:43.640805 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 438/863] 10.222.0.23 () {56 vars in 1117 bytes} [Thu May 19 11:58:43 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 11:58:43.647567 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519316f1-dfd0-4a36-9544-1760613d2814 None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:43.711116 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:58:43.772881 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519316f1-dfd0-4a36-9544-1760613d2814 None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:43.811683 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf49b69-dcb9-408f-9288-090ebba4b510 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:43.812334 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf49b69-dcb9-408f-9288-090ebba4b510 None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:43.812625 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf49b69-dcb9-408f-9288-090ebba4b510 None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:43.812942 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf49b69-dcb9-408f-9288-090ebba4b510 None admin] PATH_INFO: `/v3/users` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:43.813894 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf49b69-dcb9-408f-9288-090ebba4b510 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 11:58:43.814215 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf49b69-dcb9-408f-9288-090ebba4b510 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:43.815837 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf49b69-dcb9-408f-9288-090ebba4b510 None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:43.836212 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 426/864] 10.222.0.23 () {56 vars in 1149 bytes} [Thu May 19 11:58:43 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:58:43.844924 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff13267-d3c0-4fcb-9766-fa6dca7da0b4 None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:44.000432 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:58:44.009613 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff13267-d3c0-4fcb-9766-fa6dca7da0b4 None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:44.048446 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7044db99-76f7-499b-949b-3c15100f45e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:44.049113 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7044db99-76f7-499b-949b-3c15100f45e6 None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:44.049588 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7044db99-76f7-499b-949b-3c15100f45e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:44.050078 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7044db99-76f7-499b-949b-3c15100f45e6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:44.126951 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47083590-9e18-4b99-96f7-d6649500cfd5 None None] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:44.127280 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47083590-9e18-4b99-96f7-d6649500cfd5 None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:44.127523 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47083590-9e18-4b99-96f7-d6649500cfd5 None None] PATH_INFO: `/` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:44.129246 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 427/865] 10.222.0.23 () {54 vars in 926 bytes} [Thu May 19 11:58:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 11:58:44.131202 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7044db99-76f7-499b-949b-3c15100f45e6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:44.132728 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7044db99-76f7-499b-949b-3c15100f45e6 None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:44.134805 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9927923-23d5-42e2-9ce5-74ccc274d222 None None] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:44.135108 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9927923-23d5-42e2-9ce5-74ccc274d222 None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:44.135392 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9927923-23d5-42e2-9ce5-74ccc274d222 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:44.156705 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.auth.core [None req-c9927923-23d5-42e2-9ce5-74ccc274d222 None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 11:58:44.157935 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-c9927923-23d5-42e2-9ce5-74ccc274d222 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 11:58:44.248836 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 428/866] 10.222.0.23 () {58 vars in 1023 bytes} [Thu May 19 11:58:44 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 11:58:44.272146 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:58:44.274547 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9927923-23d5-42e2-9ce5-74ccc274d222 None None] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:44.280400 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 439/867] 10.222.0.23 () {58 vars in 1282 bytes} [Thu May 19 11:58:43 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 436 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 11:58:44.297982 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7044db99-76f7-499b-949b-3c15100f45e6 None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:44.330550 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:58:44.411022 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7044db99-76f7-499b-949b-3c15100f45e6 None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:44.416154 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9927923-23d5-42e2-9ce5-74ccc274d222 None None] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:44.458865 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6880c1cb-0682-4f97-b303-20ce86ef4d81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:44.460264 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6880c1cb-0682-4f97-b303-20ce86ef4d81 None admin] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:44.460755 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6880c1cb-0682-4f97-b303-20ce86ef4d81 None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:44.460862 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2066480-5ee1-4967-95c7-8366b4b39754 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:44.461364 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6880c1cb-0682-4f97-b303-20ce86ef4d81 None admin] PATH_INFO: `/v3/services` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:44.461617 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2066480-5ee1-4967-95c7-8366b4b39754 None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:44.461905 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2066480-5ee1-4967-95c7-8366b4b39754 None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:44.462182 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2066480-5ee1-4967-95c7-8366b4b39754 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:44.463120 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6880c1cb-0682-4f97-b303-20ce86ef4d81 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:44.465502 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6880c1cb-0682-4f97-b303-20ce86ef4d81 None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:44.473435 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2066480-5ee1-4967-95c7-8366b4b39754 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:44.475003 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2066480-5ee1-4967-95c7-8366b4b39754 None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:44.495549 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application [None req-a2066480-5ee1-4967-95c7-8366b4b39754 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 11:58:44.495549 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 11:58:44.495549 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 11:58:44.495549 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 11:58:44.495549 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 11:58:44.495549 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 11:58:44.495549 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 11:58:44.495549 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 11:58:44.495549 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 11:58:44.495549 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 11:58:44.495549 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 11:58:44.495549 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 11:58:44.495549 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 11:58:44.495549 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 11:58:44.495549 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 11:58:44.496553 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 11:58:44.496553 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 11:58:44.496553 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 11:58:44.496553 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 11:58:44.496553 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 11:58:44.496553 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 11:58:44.496553 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.get_or_create( May 19 11:58:44.496553 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 11:58:44.496553 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application with Lock( May 19 11:58:44.496553 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 11:58:44.496553 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._enter() May 19 11:58:44.496553 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 11:58:44.496553 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 11:58:44.496553 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 11:58:44.496553 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.creator() May 19 11:58:44.496553 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 11:58:44.496553 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application created_value = creator( May 19 11:58:44.497441 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 11:58:44.497441 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 11:58:44.497441 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 11:58:44.497441 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application May 19 11:58:44.497441 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 440/868] 10.222.0.23 () {56 vars in 1108 bytes} [Thu May 19 11:58:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 11:58:44.505440 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2066480-5ee1-4967-95c7-8366b4b39754 None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:44.561700 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 429/869] 10.222.0.23 () {60 vars in 1141 bytes} [Thu May 19 11:58:44 2022] POST /identity/v3/services => generated 251 bytes in 286 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 11:58:44.569909 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:58:44.629419 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2066480-5ee1-4967-95c7-8366b4b39754 None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:44.674894 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98888ec1-9220-4872-9783-e9ea93afef5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:44.675582 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98888ec1-9220-4872-9783-e9ea93afef5a None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:44.675968 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98888ec1-9220-4872-9783-e9ea93afef5a None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:44.676250 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98888ec1-9220-4872-9783-e9ea93afef5a None admin] PATH_INFO: `/v3/domains` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:44.676986 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98888ec1-9220-4872-9783-e9ea93afef5a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 11:58:44.677291 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98888ec1-9220-4872-9783-e9ea93afef5a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:44.678704 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98888ec1-9220-4872-9783-e9ea93afef5a None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:44.692952 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 441/870] 10.222.0.23 () {56 vars in 1109 bytes} [Thu May 19 11:58:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 11:58:44.696871 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eed52a3-7b82-4665-8087-f8531fe4770b None None] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:44.697168 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eed52a3-7b82-4665-8087-f8531fe4770b None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:44.697452 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eed52a3-7b82-4665-8087-f8531fe4770b None None] PATH_INFO: `/` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:44.698709 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98888ec1-9220-4872-9783-e9ea93afef5a None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:44.699178 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 430/871] 10.222.0.23 () {54 vars in 926 bytes} [Thu May 19 11:58:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 11:58:44.704897 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f8d27a-275c-46e3-9ace-8fc4c235b7aa None None] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:44.705253 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f8d27a-275c-46e3-9ace-8fc4c235b7aa None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:44.705471 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f8d27a-275c-46e3-9ace-8fc4c235b7aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:44.724499 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.auth.core [None req-d3f8d27a-275c-46e3-9ace-8fc4c235b7aa None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 11:58:44.725804 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-d3f8d27a-275c-46e3-9ace-8fc4c235b7aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 11:58:44.820507 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 431/872] 10.222.0.23 () {58 vars in 1023 bytes} [Thu May 19 11:58:44 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 11:58:44.835753 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98888ec1-9220-4872-9783-e9ea93afef5a None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:44.840546 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:58:44.841032 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f8d27a-275c-46e3-9ace-8fc4c235b7aa None None] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:44.874900 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:58:44.880225 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1cb1c22-b035-44f0-9e38-0a4f3d28926e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:44.881992 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cb1c22-b035-44f0-9e38-0a4f3d28926e None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:44.882433 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cb1c22-b035-44f0-9e38-0a4f3d28926e None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:44.882858 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cb1c22-b035-44f0-9e38-0a4f3d28926e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:44.892920 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1cb1c22-b035-44f0-9e38-0a4f3d28926e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:44.894413 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1cb1c22-b035-44f0-9e38-0a4f3d28926e None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:44.914494 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application [None req-c1cb1c22-b035-44f0-9e38-0a4f3d28926e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 11:58:44.914494 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 11:58:44.914494 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 11:58:44.914494 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 11:58:44.914494 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 11:58:44.914494 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 11:58:44.914494 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 11:58:44.914494 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 11:58:44.914494 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 11:58:44.914494 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 11:58:44.914494 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 11:58:44.914494 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 11:58:44.914494 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 11:58:44.914494 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 11:58:44.914494 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 11:58:44.921816 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 11:58:44.921816 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 11:58:44.921816 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 11:58:44.921816 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 11:58:44.921816 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 11:58:44.921816 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 11:58:44.921816 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.get_or_create( May 19 11:58:44.921816 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 11:58:44.921816 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application with Lock( May 19 11:58:44.921816 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 11:58:44.921816 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._enter() May 19 11:58:44.921816 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 11:58:44.921816 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 11:58:44.921816 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 11:58:44.921816 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.creator() May 19 11:58:44.921816 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 11:58:44.921816 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application created_value = creator( May 19 11:58:44.922717 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 11:58:44.922717 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 11:58:44.922717 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 11:58:44.922717 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 11:58:44.922717 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 11:58:44.922717 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 11:58:44.922717 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 11:58:44.922717 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application May 19 11:58:44.922717 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 442/873] 10.222.0.23 () {56 vars in 1111 bytes} [Thu May 19 11:58:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 11:58:44.926089 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1cb1c22-b035-44f0-9e38-0a4f3d28926e None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:44.962108 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f8d27a-275c-46e3-9ace-8fc4c235b7aa None None] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:45.000450 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6781eeeb-240c-4e2f-bcd2-82f6d356f273 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:45.001253 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6781eeeb-240c-4e2f-bcd2-82f6d356f273 None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:45.001532 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6781eeeb-240c-4e2f-bcd2-82f6d356f273 None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:45.001801 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6781eeeb-240c-4e2f-bcd2-82f6d356f273 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:45.002563 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6781eeeb-240c-4e2f-bcd2-82f6d356f273 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:45.004061 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6781eeeb-240c-4e2f-bcd2-82f6d356f273 None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:45.017545 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application [None req-6781eeeb-240c-4e2f-bcd2-82f6d356f273 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 11:58:45.017545 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 11:58:45.017545 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 11:58:45.017545 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 11:58:45.017545 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 11:58:45.017545 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 11:58:45.017545 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 11:58:45.017545 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 11:58:45.017545 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 11:58:45.017545 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application May 19 11:58:45.017545 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 11:58:45.017545 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application May 19 11:58:45.017545 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 11:58:45.017545 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 11:58:45.017545 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 11:58:45.017545 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 11:58:45.017545 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 11:58:45.018571 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 11:58:45.018571 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 11:58:45.018571 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 11:58:45.018571 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 11:58:45.018571 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 11:58:45.018571 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 11:58:45.018571 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 11:58:45.018571 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 11:58:45.018571 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 11:58:45.018571 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 11:58:45.018571 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 11:58:45.018571 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 11:58:45.018571 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 11:58:45.018571 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 11:58:45.018571 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 11:58:45.018571 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.get_or_create( May 19 11:58:45.019411 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 11:58:45.019411 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application with Lock( May 19 11:58:45.019411 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 11:58:45.019411 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._enter() May 19 11:58:45.019411 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 11:58:45.019411 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 11:58:45.019411 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 11:58:45.019411 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.creator() May 19 11:58:45.019411 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 11:58:45.019411 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application created_value = creator( May 19 11:58:45.019411 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 11:58:45.019411 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 11:58:45.019411 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 11:58:45.019411 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application May 19 11:58:45.019411 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 432/874] 10.222.0.23 () {56 vars in 1126 bytes} [Thu May 19 11:58:44 2022] GET /identity/v3/services/object-store => generated 93 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 11:58:45.020259 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f01634-08de-4897-94f9-4823b1c013ba None None] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:45.020329 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f01634-08de-4897-94f9-4823b1c013ba None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:45.020968 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f01634-08de-4897-94f9-4823b1c013ba None None] PATH_INFO: `/` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:45.021968 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 433/875] 10.222.0.23 () {54 vars in 926 bytes} [Thu May 19 11:58:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 11:58:45.026736 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f01634-08de-4897-94f9-4823b1c013ba None None] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:45.041513 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1cb1c22-b035-44f0-9e38-0a4f3d28926e None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:45.078847 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8651696e-4d03-401b-968e-0475e00cbf0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:45.079438 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8651696e-4d03-401b-968e-0475e00cbf0e None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:45.079725 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8651696e-4d03-401b-968e-0475e00cbf0e None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:45.080075 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8651696e-4d03-401b-968e-0475e00cbf0e None admin] PATH_INFO: `/v3/projects` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:45.080859 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8651696e-4d03-401b-968e-0475e00cbf0e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 11:58:45.081192 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8651696e-4d03-401b-968e-0475e00cbf0e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:45.082653 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8651696e-4d03-401b-968e-0475e00cbf0e None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:45.097112 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 443/876] 10.222.0.23 () {56 vars in 1148 bytes} [Thu May 19 11:58:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:58:45.098482 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7de520-985f-490f-8d30-5a506f80d920 None None] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:45.098777 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7de520-985f-490f-8d30-5a506f80d920 None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:45.099066 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7de520-985f-490f-8d30-5a506f80d920 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:45.117202 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.auth.core [None req-3a7de520-985f-490f-8d30-5a506f80d920 None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 11:58:45.118405 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-3a7de520-985f-490f-8d30-5a506f80d920 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 11:58:45.139118 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:58:45.148635 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:58:45.157565 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f01634-08de-4897-94f9-4823b1c013ba None None] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:45.195184 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99ff851-c835-482c-b996-84003f36fd9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:45.196140 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99ff851-c835-482c-b996-84003f36fd9f None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:45.203862 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99ff851-c835-482c-b996-84003f36fd9f None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:45.204067 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 444/877] 10.222.0.23 () {58 vars in 1023 bytes} [Thu May 19 11:58:45 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 11:58:45.204159 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99ff851-c835-482c-b996-84003f36fd9f None admin] PATH_INFO: `/v3/services` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:45.204242 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7de520-985f-490f-8d30-5a506f80d920 None None] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:45.204317 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99ff851-c835-482c-b996-84003f36fd9f None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 11:58:45.204317 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99ff851-c835-482c-b996-84003f36fd9f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:45.204317 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99ff851-c835-482c-b996-84003f36fd9f None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:45.215005 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 434/878] 10.222.0.23 () {56 vars in 1122 bytes} [Thu May 19 11:58:45 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:58:45.220759 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99ff851-c835-482c-b996-84003f36fd9f None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:45.311296 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7de520-985f-490f-8d30-5a506f80d920 None None] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:45.332826 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99ff851-c835-482c-b996-84003f36fd9f None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:45.349141 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a0d6b1-e707-4060-aad3-ce77d773e06b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:45.349834 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a0d6b1-e707-4060-aad3-ce77d773e06b None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:45.350105 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a0d6b1-e707-4060-aad3-ce77d773e06b None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:45.350380 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a0d6b1-e707-4060-aad3-ce77d773e06b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:45.351214 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a0d6b1-e707-4060-aad3-ce77d773e06b None admin] RBAC: Adding query filter params (role.id=7c59061ba7064a0db6032433fb0eec6b, scope.project.id=d70d25bf69314820b8aaecb65576978a, user.id=0c3181fb4f584a549a786adfefdf3dfb) {{(pid=80540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 11:58:45.351527 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a0d6b1-e707-4060-aad3-ce77d773e06b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:45.353002 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a0d6b1-e707-4060-aad3-ce77d773e06b None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:45.362699 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 445/879] 10.222.0.23 () {56 vars in 1374 bytes} [Thu May 19 11:58:45 2022] GET /identity/v3/role_assignments?user.id=0c3181fb4f584a549a786adfefdf3dfb&scope.project.id=d70d25bf69314820b8aaecb65576978a&role.id=7c59061ba7064a0db6032433fb0eec6b => generated 608 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:58:45.362699 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a0d6b1-e707-4060-aad3-ce77d773e06b None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:45.383454 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:58:45.383867 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02410772-d00b-4d92-92f6-5aa615b226d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:45.384834 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02410772-d00b-4d92-92f6-5aa615b226d1 None admin] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:45.385303 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02410772-d00b-4d92-92f6-5aa615b226d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:45.385770 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02410772-d00b-4d92-92f6-5aa615b226d1 None admin] PATH_INFO: `/v3/services` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:45.387741 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02410772-d00b-4d92-92f6-5aa615b226d1 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:45.390336 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02410772-d00b-4d92-92f6-5aa615b226d1 None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:45.479748 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a0d6b1-e707-4060-aad3-ce77d773e06b None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:45.505248 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 435/880] 10.222.0.23 () {60 vars in 1141 bytes} [Thu May 19 11:58:45 2022] POST /identity/v3/services => generated 269 bytes in 285 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 11:58:45.540330 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514f8209-54f0-4992-a73b-069d97142645 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:45.541528 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514f8209-54f0-4992-a73b-069d97142645 None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:45.542103 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514f8209-54f0-4992-a73b-069d97142645 None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:45.542596 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514f8209-54f0-4992-a73b-069d97142645 None admin] PATH_INFO: `/v3/services` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:45.543642 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514f8209-54f0-4992-a73b-069d97142645 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 11:58:45.544309 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514f8209-54f0-4992-a73b-069d97142645 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:45.546044 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514f8209-54f0-4992-a73b-069d97142645 None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:45.552876 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 446/881] 10.222.0.23 () {56 vars in 1122 bytes} [Thu May 19 11:58:45 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:58:47.186386 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882ebf1b-63e9-418e-958a-9dc3241ed0ec None None] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:47.187276 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882ebf1b-63e9-418e-958a-9dc3241ed0ec None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:47.187714 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882ebf1b-63e9-418e-958a-9dc3241ed0ec None None] PATH_INFO: `/` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:47.189537 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 436/882] 10.222.0.23 () {54 vars in 926 bytes} [Thu May 19 11:58:47 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 11:58:47.195402 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc93fee-384a-47dc-b2e4-d639c4ff9629 None None] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:47.196575 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc93fee-384a-47dc-b2e4-d639c4ff9629 None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:47.197059 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc93fee-384a-47dc-b2e4-d639c4ff9629 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:47.215434 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.auth.core [None req-2dc93fee-384a-47dc-b2e4-d639c4ff9629 None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 11:58:47.217656 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-2dc93fee-384a-47dc-b2e4-d639c4ff9629 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 11:58:47.295937 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 447/883] 10.222.0.23 () {58 vars in 1023 bytes} [Thu May 19 11:58:47 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 11:58:47.314798 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882ebf1b-63e9-418e-958a-9dc3241ed0ec None None] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:47.335459 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:58:47.426353 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882ebf1b-63e9-418e-958a-9dc3241ed0ec None None] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:47.463685 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a1485a-d126-4c04-8348-39551716196a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:47.464528 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a1485a-d126-4c04-8348-39551716196a None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:47.464812 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a1485a-d126-4c04-8348-39551716196a None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:47.465090 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a1485a-d126-4c04-8348-39551716196a None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:47.465811 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a1485a-d126-4c04-8348-39551716196a None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:47.467176 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a1485a-d126-4c04-8348-39551716196a None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:47.479397 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application [None req-e3a1485a-d126-4c04-8348-39551716196a None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. May 19 11:58:47.479397 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 11:58:47.479397 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 11:58:47.479397 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 11:58:47.479397 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 11:58:47.479397 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 11:58:47.479397 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 11:58:47.479397 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 11:58:47.479397 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 19 11:58:47.479397 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application May 19 11:58:47.479397 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 11:58:47.479397 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application May 19 11:58:47.479397 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 11:58:47.479397 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 11:58:47.479397 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 11:58:47.479397 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 11:58:47.479397 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 11:58:47.480471 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 11:58:47.480471 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 11:58:47.480471 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 11:58:47.480471 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 11:58:47.480471 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 11:58:47.480471 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 11:58:47.480471 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 11:58:47.480471 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 11:58:47.480471 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 11:58:47.480471 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 11:58:47.480471 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 11:58:47.480471 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 11:58:47.480471 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 11:58:47.480471 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 11:58:47.480471 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 11:58:47.480471 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.get_or_create( May 19 11:58:47.481302 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 11:58:47.481302 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application with Lock( May 19 11:58:47.481302 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 11:58:47.481302 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._enter() May 19 11:58:47.481302 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 11:58:47.481302 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 11:58:47.481302 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 11:58:47.481302 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.creator() May 19 11:58:47.481302 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 11:58:47.481302 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application created_value = creator( May 19 11:58:47.481302 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 11:58:47.481302 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 11:58:47.481302 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 19 11:58:47.481302 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application May 19 11:58:47.486160 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 437/884] 10.222.0.23 () {56 vars in 1114 bytes} [Thu May 19 11:58:47 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 11:58:47.487686 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc93fee-384a-47dc-b2e4-d639c4ff9629 None None] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:47.499054 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:58:47.595422 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc93fee-384a-47dc-b2e4-d639c4ff9629 None None] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:47.637175 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a14415e-ab35-4513-999e-7f761222bd82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:47.638170 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a14415e-ab35-4513-999e-7f761222bd82 None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:47.638723 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a14415e-ab35-4513-999e-7f761222bd82 None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:47.639184 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a14415e-ab35-4513-999e-7f761222bd82 None admin] PATH_INFO: `/v3/services` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:47.640140 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a14415e-ab35-4513-999e-7f761222bd82 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 11:58:47.640643 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a14415e-ab35-4513-999e-7f761222bd82 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:47.642356 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a14415e-ab35-4513-999e-7f761222bd82 None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:47.653415 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 448/885] 10.222.0.23 () {56 vars in 1114 bytes} [Thu May 19 11:58:47 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:58:47.655549 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a1485a-d126-4c04-8348-39551716196a None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:47.755636 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:58:47.775753 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a1485a-d126-4c04-8348-39551716196a None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:47.812699 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfe6331-4254-4335-a2ee-1d2b7897d5af None None] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:47.812894 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4025cd8-b03f-4844-aef8-03dfb0d06c68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:47.813026 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfe6331-4254-4335-a2ee-1d2b7897d5af None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:47.813270 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfe6331-4254-4335-a2ee-1d2b7897d5af None None] PATH_INFO: `/` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:47.813472 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4025cd8-b03f-4844-aef8-03dfb0d06c68 None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:47.813761 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4025cd8-b03f-4844-aef8-03dfb0d06c68 None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:47.814038 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4025cd8-b03f-4844-aef8-03dfb0d06c68 None admin] PATH_INFO: `/v3/services` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:47.814902 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4025cd8-b03f-4844-aef8-03dfb0d06c68 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 11:58:47.815064 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 449/886] 10.222.0.23 () {54 vars in 926 bytes} [Thu May 19 11:58:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 11:58:47.815263 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4025cd8-b03f-4844-aef8-03dfb0d06c68 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:47.816775 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4025cd8-b03f-4844-aef8-03dfb0d06c68 None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:47.820569 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f680c309-167e-41d1-a8dc-7db1da09088e None None] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:47.821129 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f680c309-167e-41d1-a8dc-7db1da09088e None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:47.821413 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f680c309-167e-41d1-a8dc-7db1da09088e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:47.823874 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 438/887] 10.222.0.23 () {56 vars in 1114 bytes} [Thu May 19 11:58:47 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:58:47.833571 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4025cd8-b03f-4844-aef8-03dfb0d06c68 None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:47.844390 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.auth.core [None req-f680c309-167e-41d1-a8dc-7db1da09088e None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 11:58:47.845630 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-f680c309-167e-41d1-a8dc-7db1da09088e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 11:58:47.918383 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 450/888] 10.222.0.23 () {58 vars in 1023 bytes} [Thu May 19 11:58:47 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 11:58:47.921219 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5d660d-4bdf-488e-98e0-bf5bc763236e None None] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:47.921708 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5d660d-4bdf-488e-98e0-bf5bc763236e None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:47.922170 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5d660d-4bdf-488e-98e0-bf5bc763236e None None] PATH_INFO: `/` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:47.924665 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 451/889] 10.222.0.23 () {54 vars in 926 bytes} [Thu May 19 11:58:47 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 11:58:47.929903 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede5001e-b374-484a-9549-9831f64fa01e None None] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:47.930454 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede5001e-b374-484a-9549-9831f64fa01e None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:47.930454 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede5001e-b374-484a-9549-9831f64fa01e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:47.940556 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:58:47.956205 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4025cd8-b03f-4844-aef8-03dfb0d06c68 None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:47.957218 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.auth.core [None req-ede5001e-b374-484a-9549-9831f64fa01e None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 11:58:47.958414 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-ede5001e-b374-484a-9549-9831f64fa01e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 11:58:47.990094 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0b91ad-c6fb-4e24-b8e0-29f5a84f68e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:47.990758 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0b91ad-c6fb-4e24-b8e0-29f5a84f68e6 None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:47.991043 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0b91ad-c6fb-4e24-b8e0-29f5a84f68e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:47.991316 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0b91ad-c6fb-4e24-b8e0-29f5a84f68e6 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:47.992220 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0b91ad-c6fb-4e24-b8e0-29f5a84f68e6 None admin] RBAC: Adding query filter params (interface=public, service_id=d55eb0387d34425688793192abdf2152, region_id=RegionOne) {{(pid=80539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 11:58:47.992536 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0b91ad-c6fb-4e24-b8e0-29f5a84f68e6 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:47.993915 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0b91ad-c6fb-4e24-b8e0-29f5a84f68e6 None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:47.999553 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 439/890] 10.222.0.23 () {56 vars in 1251 bytes} [Thu May 19 11:58:47 2022] GET /identity/v3/endpoints?service_id=d55eb0387d34425688793192abdf2152&interface=public®ion_id=RegionOne => generated 194 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:58:48.000960 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d60d7a5-f8eb-4bd6-bad4-1270fd898adb None None] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:48.001255 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d60d7a5-f8eb-4bd6-bad4-1270fd898adb None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:48.001531 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d60d7a5-f8eb-4bd6-bad4-1270fd898adb None None] PATH_INFO: `/` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:48.011120 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 440/891] 10.222.0.23 () {54 vars in 926 bytes} [Thu May 19 11:58:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 11:58:48.011120 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d60d7a5-f8eb-4bd6-bad4-1270fd898adb None None] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:48.017036 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:58:48.030876 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 452/892] 10.222.0.23 () {58 vars in 1023 bytes} [Thu May 19 11:58:47 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 11:58:48.031731 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede5001e-b374-484a-9549-9831f64fa01e None None] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:48.114244 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d60d7a5-f8eb-4bd6-bad4-1270fd898adb None None] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:48.137079 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede5001e-b374-484a-9549-9831f64fa01e None None] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:48.150503 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df583dc-9aae-4e25-b6ac-a6572b5765c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:48.151328 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df583dc-9aae-4e25-b6ac-a6572b5765c3 None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:48.151605 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df583dc-9aae-4e25-b6ac-a6572b5765c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:48.151907 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df583dc-9aae-4e25-b6ac-a6572b5765c3 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:48.163461 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df583dc-9aae-4e25-b6ac-a6572b5765c3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 11:58:48.169081 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:58:48.171961 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application [None req-7df583dc-9aae-4e25-b6ac-a6572b5765c3 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 19 11:58:48.171961 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 11:58:48.171961 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 11:58:48.171961 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 11:58:48.171961 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 11:58:48.171961 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 11:58:48.171961 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 11:58:48.171961 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 11:58:48.171961 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 11:58:48.171961 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 11:58:48.171961 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 11:58:48.171961 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 11:58:48.171961 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 11:58:48.171961 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 11:58:48.171961 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 11:58:48.171961 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 11:58:48.173061 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 11:58:48.173061 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 11:58:48.173061 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 11:58:48.173061 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 11:58:48.173061 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 11:58:48.173061 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 11:58:48.173061 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 11:58:48.173061 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 11:58:48.173061 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 11:58:48.173061 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.get_or_create( May 19 11:58:48.173061 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 11:58:48.173061 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application with Lock( May 19 11:58:48.173061 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 11:58:48.173061 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._enter() May 19 11:58:48.173061 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 11:58:48.173061 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 11:58:48.173908 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 11:58:48.173908 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.creator() May 19 11:58:48.173908 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 11:58:48.173908 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application created_value = creator( May 19 11:58:48.173908 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 11:58:48.173908 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 11:58:48.173908 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 11:58:48.173908 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 11:58:48.173908 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 11:58:48.173908 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 11:58:48.173908 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 19 11:58:48.173908 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application May 19 11:58:48.173908 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 441/893] 10.222.0.23 () {56 vars in 1120 bytes} [Thu May 19 11:58:48 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 11:58:48.174693 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639dccd8-e49a-4e5b-a004-1e24d9ea9b12 None None] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:48.174693 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639dccd8-e49a-4e5b-a004-1e24d9ea9b12 None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:48.174918 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639dccd8-e49a-4e5b-a004-1e24d9ea9b12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:48.178682 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb115d3-8de2-483b-aed5-d3953da2b780 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:48.179360 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb115d3-8de2-483b-aed5-d3953da2b780 None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:48.179647 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb115d3-8de2-483b-aed5-d3953da2b780 None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:48.183607 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb115d3-8de2-483b-aed5-d3953da2b780 None admin] PATH_INFO: `/v3/services` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:48.183607 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb115d3-8de2-483b-aed5-d3953da2b780 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:48.183607 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb115d3-8de2-483b-aed5-d3953da2b780 None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:48.188638 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 453/894] 10.222.0.23 () {56 vars in 1087 bytes} [Thu May 19 11:58:48 2022] GET /identity/v3/services => generated 1268 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 11:58:48.189403 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb115d3-8de2-483b-aed5-d3953da2b780 None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:48.194550 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.auth.core [None req-639dccd8-e49a-4e5b-a004-1e24d9ea9b12 None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 11:58:48.196106 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-639dccd8-e49a-4e5b-a004-1e24d9ea9b12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 11:58:48.276999 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:58:48.283973 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 442/895] 10.222.0.23 () {58 vars in 1023 bytes} [Thu May 19 11:58:48 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 11:58:48.285415 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639dccd8-e49a-4e5b-a004-1e24d9ea9b12 None None] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:48.321442 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb115d3-8de2-483b-aed5-d3953da2b780 None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:48.365287 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688d66fe-0654-4841-96d3-6d0fdd669b93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:48.366001 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688d66fe-0654-4841-96d3-6d0fdd669b93 None admin] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:48.366296 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688d66fe-0654-4841-96d3-6d0fdd669b93 None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:48.366582 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688d66fe-0654-4841-96d3-6d0fdd669b93 None admin] PATH_INFO: `/v3/services` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:48.367315 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688d66fe-0654-4841-96d3-6d0fdd669b93 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:48.368772 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688d66fe-0654-4841-96d3-6d0fdd669b93 None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:48.414730 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639dccd8-e49a-4e5b-a004-1e24d9ea9b12 None None] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:48.433425 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 454/896] 10.222.0.23 () {60 vars in 1141 bytes} [Thu May 19 11:58:48 2022] POST /identity/v3/services => generated 241 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 11:58:48.434585 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688d66fe-0654-4841-96d3-6d0fdd669b93 None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:48.475005 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5fd52a-16e3-435d-94b8-0ff85c3817cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:48.475693 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5fd52a-16e3-435d-94b8-0ff85c3817cb None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:48.476181 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5fd52a-16e3-435d-94b8-0ff85c3817cb None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:48.476464 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5fd52a-16e3-435d-94b8-0ff85c3817cb None admin] PATH_INFO: `/v3/roles` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:48.477554 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5fd52a-16e3-435d-94b8-0ff85c3817cb None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 11:58:48.477910 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5fd52a-16e3-435d-94b8-0ff85c3817cb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:48.479992 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5fd52a-16e3-435d-94b8-0ff85c3817cb None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:48.492214 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 443/897] 10.222.0.23 () {56 vars in 1115 bytes} [Thu May 19 11:58:48 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:58:48.498595 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5fd52a-16e3-435d-94b8-0ff85c3817cb None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:48.530417 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:58:48.591067 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688d66fe-0654-4841-96d3-6d0fdd669b93 None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:48.596377 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:58:48.650297 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5fd52a-16e3-435d-94b8-0ff85c3817cb None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:48.652360 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32b6a8f-21d7-4bc7-abae-7c3703efb163 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:48.653183 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32b6a8f-21d7-4bc7-abae-7c3703efb163 None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:48.653465 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32b6a8f-21d7-4bc7-abae-7c3703efb163 None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:48.653739 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32b6a8f-21d7-4bc7-abae-7c3703efb163 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:48.654454 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32b6a8f-21d7-4bc7-abae-7c3703efb163 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:48.655876 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32b6a8f-21d7-4bc7-abae-7c3703efb163 None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:48.671263 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application [None req-d32b6a8f-21d7-4bc7-abae-7c3703efb163 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 19 11:58:48.671263 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 11:58:48.671263 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 11:58:48.671263 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 11:58:48.671263 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 11:58:48.671263 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 11:58:48.671263 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 11:58:48.671263 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 11:58:48.671263 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 19 11:58:48.671263 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application May 19 11:58:48.671263 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 11:58:48.671263 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application May 19 11:58:48.671263 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 11:58:48.671263 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 11:58:48.671263 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 11:58:48.671263 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 11:58:48.672514 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 11:58:48.672514 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 11:58:48.672514 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 11:58:48.672514 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 11:58:48.672514 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 11:58:48.672514 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 11:58:48.672514 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 11:58:48.672514 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 11:58:48.672514 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 11:58:48.672514 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 11:58:48.672514 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 11:58:48.672514 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 11:58:48.672514 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 11:58:48.672514 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 11:58:48.672514 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 11:58:48.672514 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 11:58:48.673344 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.get_or_create( May 19 11:58:48.673344 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 11:58:48.673344 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application with Lock( May 19 11:58:48.673344 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 11:58:48.673344 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._enter() May 19 11:58:48.673344 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 11:58:48.673344 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 11:58:48.673344 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 11:58:48.673344 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.creator() May 19 11:58:48.673344 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 11:58:48.673344 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application created_value = creator( May 19 11:58:48.673344 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 11:58:48.673344 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 11:58:48.673344 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 19 11:58:48.673344 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application May 19 11:58:48.673344 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 455/898] 10.222.0.23 () {56 vars in 1132 bytes} [Thu May 19 11:58:48 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 11:58:48.680708 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32b6a8f-21d7-4bc7-abae-7c3703efb163 None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:48.691756 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0559a94e-cfb7-4379-8209-d55e40416d98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:48.692590 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0559a94e-cfb7-4379-8209-d55e40416d98 None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:48.692872 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0559a94e-cfb7-4379-8209-d55e40416d98 None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:48.693143 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0559a94e-cfb7-4379-8209-d55e40416d98 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:48.707124 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0559a94e-cfb7-4379-8209-d55e40416d98 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:48.708695 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0559a94e-cfb7-4379-8209-d55e40416d98 None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:48.730917 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application [None req-0559a94e-cfb7-4379-8209-d55e40416d98 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 11:58:48.730917 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 11:58:48.730917 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 11:58:48.730917 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 11:58:48.730917 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 11:58:48.730917 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 11:58:48.730917 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 11:58:48.730917 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 11:58:48.730917 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 11:58:48.730917 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 11:58:48.730917 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 11:58:48.730917 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 11:58:48.730917 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 11:58:48.730917 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 11:58:48.730917 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 11:58:48.732038 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 11:58:48.732038 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 11:58:48.732038 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 11:58:48.732038 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 11:58:48.732038 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 11:58:48.732038 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 11:58:48.732038 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.get_or_create( May 19 11:58:48.732038 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 11:58:48.732038 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application with Lock( May 19 11:58:48.732038 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 11:58:48.732038 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._enter() May 19 11:58:48.732038 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 11:58:48.732038 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 11:58:48.732038 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 11:58:48.732038 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.creator() May 19 11:58:48.732038 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 11:58:48.732038 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application created_value = creator( May 19 11:58:48.732978 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 11:58:48.732978 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 11:58:48.732978 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 11:58:48.732978 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application May 19 11:58:48.733923 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 444/899] 10.222.0.23 () {56 vars in 1108 bytes} [Thu May 19 11:58:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 11:58:48.738942 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0559a94e-cfb7-4379-8209-d55e40416d98 None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:48.814977 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32b6a8f-21d7-4bc7-abae-7c3703efb163 None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:48.842185 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:58:48.847808 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0559a94e-cfb7-4379-8209-d55e40416d98 None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:48.859070 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:58:48.860575 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7348c26-fd80-4157-bbed-f9b60a0eb88a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:48.861602 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7348c26-fd80-4157-bbed-f9b60a0eb88a None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:48.862070 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7348c26-fd80-4157-bbed-f9b60a0eb88a None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:48.862589 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7348c26-fd80-4157-bbed-f9b60a0eb88a None admin] PATH_INFO: `/v3/services` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:48.863518 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7348c26-fd80-4157-bbed-f9b60a0eb88a None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 11:58:48.864029 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7348c26-fd80-4157-bbed-f9b60a0eb88a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:48.866249 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7348c26-fd80-4157-bbed-f9b60a0eb88a None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:48.881174 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 456/900] 10.222.0.23 () {56 vars in 1126 bytes} [Thu May 19 11:58:48 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:58:48.883082 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7348c26-fd80-4157-bbed-f9b60a0eb88a None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:48.896680 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7291c99-9041-4887-b4aa-77426ba94665 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:48.897996 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7291c99-9041-4887-b4aa-77426ba94665 None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:48.898383 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7291c99-9041-4887-b4aa-77426ba94665 None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:48.898765 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7291c99-9041-4887-b4aa-77426ba94665 None admin] PATH_INFO: `/v3/domains` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:48.900067 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7291c99-9041-4887-b4aa-77426ba94665 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 11:58:48.900785 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7291c99-9041-4887-b4aa-77426ba94665 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:48.902533 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7291c99-9041-4887-b4aa-77426ba94665 None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:48.925127 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 445/901] 10.222.0.23 () {56 vars in 1109 bytes} [Thu May 19 11:58:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 11:58:48.926538 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7291c99-9041-4887-b4aa-77426ba94665 None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:49.016061 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7348c26-fd80-4157-bbed-f9b60a0eb88a None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:49.056291 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c22e9d-134c-4f16-81d2-d27335d87d80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:49.057322 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c22e9d-134c-4f16-81d2-d27335d87d80 None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:49.057790 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7291c99-9041-4887-b4aa-77426ba94665 None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:49.058068 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c22e9d-134c-4f16-81d2-d27335d87d80 None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:49.058556 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c22e9d-134c-4f16-81d2-d27335d87d80 None admin] PATH_INFO: `/v3/services` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:49.059576 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c22e9d-134c-4f16-81d2-d27335d87d80 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 11:58:49.060787 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c22e9d-134c-4f16-81d2-d27335d87d80 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:49.062590 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c22e9d-134c-4f16-81d2-d27335d87d80 None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:49.073746 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 457/902] 10.222.0.23 () {56 vars in 1126 bytes} [Thu May 19 11:58:48 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:58:49.075106 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c22e9d-134c-4f16-81d2-d27335d87d80 None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:49.097232 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945248e2-4a3d-4bb8-8fbe-5f64cdf5a080 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:49.098404 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945248e2-4a3d-4bb8-8fbe-5f64cdf5a080 None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:49.098922 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945248e2-4a3d-4bb8-8fbe-5f64cdf5a080 None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:49.099478 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945248e2-4a3d-4bb8-8fbe-5f64cdf5a080 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:49.108509 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945248e2-4a3d-4bb8-8fbe-5f64cdf5a080 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:49.110335 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945248e2-4a3d-4bb8-8fbe-5f64cdf5a080 None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:49.127554 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application [None req-945248e2-4a3d-4bb8-8fbe-5f64cdf5a080 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 19 11:58:49.127554 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 11:58:49.127554 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 11:58:49.127554 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 11:58:49.127554 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 11:58:49.127554 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 11:58:49.127554 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 11:58:49.127554 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 11:58:49.127554 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 11:58:49.127554 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 11:58:49.127554 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 11:58:49.127554 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 11:58:49.127554 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 11:58:49.127554 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 11:58:49.127554 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 11:58:49.127554 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 11:58:49.128590 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 11:58:49.128590 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 11:58:49.128590 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 11:58:49.128590 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 11:58:49.128590 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 11:58:49.128590 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 11:58:49.128590 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 11:58:49.128590 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 11:58:49.128590 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 11:58:49.128590 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.get_or_create( May 19 11:58:49.128590 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 11:58:49.128590 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application with Lock( May 19 11:58:49.128590 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 11:58:49.128590 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._enter() May 19 11:58:49.128590 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 11:58:49.128590 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 11:58:49.128590 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 11:58:49.128590 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.creator() May 19 11:58:49.129510 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 11:58:49.129510 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application created_value = creator( May 19 11:58:49.129510 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 11:58:49.129510 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 11:58:49.129510 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 11:58:49.129510 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 11:58:49.129510 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 11:58:49.129510 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 11:58:49.129510 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 19 11:58:49.129510 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application May 19 11:58:49.136213 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:58:49.137059 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 446/903] 10.222.0.23 () {56 vars in 1117 bytes} [Thu May 19 11:58:48 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 11:58:49.138097 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945248e2-4a3d-4bb8-8fbe-5f64cdf5a080 None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:49.171267 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:58:49.199775 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c22e9d-134c-4f16-81d2-d27335d87d80 None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:49.240472 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61cba02c-141c-4d5f-b0da-fda7edc48d59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:49.241554 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cba02c-141c-4d5f-b0da-fda7edc48d59 None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:49.242182 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cba02c-141c-4d5f-b0da-fda7edc48d59 None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:49.242694 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cba02c-141c-4d5f-b0da-fda7edc48d59 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:49.243801 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61cba02c-141c-4d5f-b0da-fda7edc48d59 None admin] RBAC: Adding query filter params (interface=public, service_id=ab30561fa55b41b59a6396750b1850e4, region_id=RegionOne) {{(pid=80540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 11:58:49.244353 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61cba02c-141c-4d5f-b0da-fda7edc48d59 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:49.246085 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61cba02c-141c-4d5f-b0da-fda7edc48d59 None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:49.257028 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 458/904] 10.222.0.23 () {56 vars in 1251 bytes} [Thu May 19 11:58:49 2022] GET /identity/v3/endpoints?service_id=ab30561fa55b41b59a6396750b1850e4&interface=public®ion_id=RegionOne => generated 194 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:58:49.258265 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61cba02c-141c-4d5f-b0da-fda7edc48d59 None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:49.259181 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945248e2-4a3d-4bb8-8fbe-5f64cdf5a080 None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:49.305351 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3e7312-58f5-4fc5-9d48-3780a082d1ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:49.306540 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3e7312-58f5-4fc5-9d48-3780a082d1ef None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:49.306833 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3e7312-58f5-4fc5-9d48-3780a082d1ef None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:49.307116 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3e7312-58f5-4fc5-9d48-3780a082d1ef None admin] PATH_INFO: `/v3/users` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:49.307995 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3e7312-58f5-4fc5-9d48-3780a082d1ef None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 11:58:49.308308 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3e7312-58f5-4fc5-9d48-3780a082d1ef None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:49.310849 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3e7312-58f5-4fc5-9d48-3780a082d1ef None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:49.338040 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 447/905] 10.222.0.23 () {56 vars in 1149 bytes} [Thu May 19 11:58:49 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:58:49.339371 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3e7312-58f5-4fc5-9d48-3780a082d1ef None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:49.391038 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61cba02c-141c-4d5f-b0da-fda7edc48d59 None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:49.405822 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:58:49.436030 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9a0bc3-e264-4eb1-9167-d03ad98d319b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:49.436713 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9a0bc3-e264-4eb1-9167-d03ad98d319b None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:49.437163 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9a0bc3-e264-4eb1-9167-d03ad98d319b None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:49.437443 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9a0bc3-e264-4eb1-9167-d03ad98d319b None admin] PATH_INFO: `/v3/services` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:49.438539 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9a0bc3-e264-4eb1-9167-d03ad98d319b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:49.440239 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9a0bc3-e264-4eb1-9167-d03ad98d319b None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:49.448893 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 459/906] 10.222.0.23 () {56 vars in 1087 bytes} [Thu May 19 11:58:49 2022] GET /identity/v3/services => generated 1497 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 11:58:49.456269 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:58:49.462385 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3e7312-58f5-4fc5-9d48-3780a082d1ef None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:49.503957 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1adcade1-f395-4c53-bc04-811c7dd1b431 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:49.504622 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adcade1-f395-4c53-bc04-811c7dd1b431 None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:49.504909 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adcade1-f395-4c53-bc04-811c7dd1b431 None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:49.505185 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adcade1-f395-4c53-bc04-811c7dd1b431 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:49.584053 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1adcade1-f395-4c53-bc04-811c7dd1b431 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:49.585541 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1adcade1-f395-4c53-bc04-811c7dd1b431 None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:49.685949 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:58:49.699599 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 448/907] 10.222.0.23 () {58 vars in 1282 bytes} [Thu May 19 11:58:49 2022] GET /identity/v3/auth/tokens => generated 1464 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 11:58:49.706258 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9a0bc3-e264-4eb1-9167-d03ad98d319b None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:49.812984 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9a0bc3-e264-4eb1-9167-d03ad98d319b None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:49.853931 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2926b76a-f5bb-48a2-adb5-09febe087133 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:49.854731 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2926b76a-f5bb-48a2-adb5-09febe087133 None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:49.855021 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2926b76a-f5bb-48a2-adb5-09febe087133 None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:49.855304 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2926b76a-f5bb-48a2-adb5-09febe087133 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:49.867309 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2926b76a-f5bb-48a2-adb5-09febe087133 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:49.868830 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2926b76a-f5bb-48a2-adb5-09febe087133 None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:49.887337 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application [None req-2926b76a-f5bb-48a2-adb5-09febe087133 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 11:58:49.887337 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 11:58:49.887337 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 11:58:49.887337 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 11:58:49.887337 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 11:58:49.887337 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 11:58:49.887337 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 11:58:49.887337 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 11:58:49.887337 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 11:58:49.887337 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 11:58:49.887337 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 11:58:49.887337 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 11:58:49.887337 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 11:58:49.887337 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 11:58:49.887337 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 11:58:49.888376 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 11:58:49.888376 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 11:58:49.888376 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 11:58:49.888376 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 11:58:49.888376 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 11:58:49.888376 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 11:58:49.888376 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.get_or_create( May 19 11:58:49.888376 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 11:58:49.888376 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application with Lock( May 19 11:58:49.888376 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 11:58:49.888376 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._enter() May 19 11:58:49.888376 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 11:58:49.888376 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 11:58:49.888376 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 11:58:49.888376 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.creator() May 19 11:58:49.888376 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 11:58:49.888376 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application created_value = creator( May 19 11:58:49.890018 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 11:58:49.890018 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 11:58:49.890018 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 11:58:49.890018 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application May 19 11:58:49.890018 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 460/908] 10.222.0.23 () {56 vars in 1108 bytes} [Thu May 19 11:58:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 11:58:49.894264 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1adcade1-f395-4c53-bc04-811c7dd1b431 None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:50.008444 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1adcade1-f395-4c53-bc04-811c7dd1b431 None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:50.044303 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b3a819-3876-411e-8ca6-dd29bfedaed6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:50.045586 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b3a819-3876-411e-8ca6-dd29bfedaed6 None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:50.045874 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b3a819-3876-411e-8ca6-dd29bfedaed6 None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:50.046156 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b3a819-3876-411e-8ca6-dd29bfedaed6 None admin] PATH_INFO: `/v3/domains` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:50.046893 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b3a819-3876-411e-8ca6-dd29bfedaed6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 11:58:50.047202 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b3a819-3876-411e-8ca6-dd29bfedaed6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:50.048611 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b3a819-3876-411e-8ca6-dd29bfedaed6 None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:50.062543 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 449/909] 10.222.0.23 () {56 vars in 1109 bytes} [Thu May 19 11:58:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 11:58:50.067925 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2926b76a-f5bb-48a2-adb5-09febe087133 None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:50.104222 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:58:50.179648 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2926b76a-f5bb-48a2-adb5-09febe087133 None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:50.217361 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc004d2-1632-471b-b2ec-5dbbee13f601 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:50.218090 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc004d2-1632-471b-b2ec-5dbbee13f601 None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:50.218378 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc004d2-1632-471b-b2ec-5dbbee13f601 None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:50.218658 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc004d2-1632-471b-b2ec-5dbbee13f601 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:50.226080 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc004d2-1632-471b-b2ec-5dbbee13f601 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:50.227571 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc004d2-1632-471b-b2ec-5dbbee13f601 None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:50.242311 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application [None req-bcc004d2-1632-471b-b2ec-5dbbee13f601 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 11:58:50.242311 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 11:58:50.242311 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 11:58:50.242311 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 11:58:50.242311 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 11:58:50.242311 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 11:58:50.242311 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 11:58:50.242311 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 11:58:50.242311 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 11:58:50.242311 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 11:58:50.242311 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 11:58:50.242311 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 11:58:50.242311 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 11:58:50.242311 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 11:58:50.242311 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 11:58:50.244248 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 11:58:50.244248 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 11:58:50.244248 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 11:58:50.244248 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 11:58:50.244248 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 11:58:50.244248 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 11:58:50.244248 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.get_or_create( May 19 11:58:50.244248 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 11:58:50.244248 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application with Lock( May 19 11:58:50.244248 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 11:58:50.244248 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._enter() May 19 11:58:50.244248 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 11:58:50.244248 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 11:58:50.244248 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 11:58:50.244248 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.creator() May 19 11:58:50.244248 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 11:58:50.244248 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application created_value = creator( May 19 11:58:50.245605 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 11:58:50.245605 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 11:58:50.245605 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 11:58:50.245605 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 11:58:50.245605 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 11:58:50.245605 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 11:58:50.245605 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 11:58:50.245605 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application May 19 11:58:50.245605 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 461/910] 10.222.0.23 () {56 vars in 1111 bytes} [Thu May 19 11:58:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 11:58:50.250013 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b3a819-3876-411e-8ca6-dd29bfedaed6 None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:50.304942 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:58:50.369789 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b3a819-3876-411e-8ca6-dd29bfedaed6 None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:50.408528 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22bdadfb-9f94-4cd5-b1d7-9956df9a335d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:50.409202 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bdadfb-9f94-4cd5-b1d7-9956df9a335d None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:50.409483 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bdadfb-9f94-4cd5-b1d7-9956df9a335d None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:50.409760 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bdadfb-9f94-4cd5-b1d7-9956df9a335d None admin] PATH_INFO: `/v3/projects` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:50.410558 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22bdadfb-9f94-4cd5-b1d7-9956df9a335d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 11:58:50.410872 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22bdadfb-9f94-4cd5-b1d7-9956df9a335d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:50.412344 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22bdadfb-9f94-4cd5-b1d7-9956df9a335d None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:50.425565 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 450/911] 10.222.0.23 () {56 vars in 1148 bytes} [Thu May 19 11:58:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:58:50.432894 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc004d2-1632-471b-b2ec-5dbbee13f601 None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:50.532262 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:58:50.537341 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a96adf8-f78a-4bcf-8a1b-9cc4c7617156 None None] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:50.537841 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a96adf8-f78a-4bcf-8a1b-9cc4c7617156 None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:50.538183 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a96adf8-f78a-4bcf-8a1b-9cc4c7617156 None None] PATH_INFO: `/` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:50.540496 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 451/912] 10.222.0.23 () {54 vars in 926 bytes} [Thu May 19 11:58:50 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 11:58:50.547075 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3595c42a-0072-48dc-9676-86d36c4fdbe5 None None] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:50.547626 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3595c42a-0072-48dc-9676-86d36c4fdbe5 None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:50.548121 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3595c42a-0072-48dc-9676-86d36c4fdbe5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:50.549707 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc004d2-1632-471b-b2ec-5dbbee13f601 None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:50.569948 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.auth.core [None req-3595c42a-0072-48dc-9676-86d36c4fdbe5 None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 11:58:50.571554 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-3595c42a-0072-48dc-9676-86d36c4fdbe5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 11:58:50.587682 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34098951-0473-4101-a992-37289887456a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:50.588413 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34098951-0473-4101-a992-37289887456a None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:50.588702 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34098951-0473-4101-a992-37289887456a None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:50.588981 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34098951-0473-4101-a992-37289887456a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:50.589845 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34098951-0473-4101-a992-37289887456a None admin] RBAC: Adding query filter params (role.id=ff2dfc5f80304d9cbd3320d8ceffe2a2, scope.project.id=d70d25bf69314820b8aaecb65576978a, user.id=0c3181fb4f584a549a786adfefdf3dfb) {{(pid=80540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 11:58:50.590160 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34098951-0473-4101-a992-37289887456a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:50.591907 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34098951-0473-4101-a992-37289887456a None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:50.599038 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 462/913] 10.222.0.23 () {56 vars in 1374 bytes} [Thu May 19 11:58:50 2022] GET /identity/v3/role_assignments?user.id=0c3181fb4f584a549a786adfefdf3dfb&scope.project.id=d70d25bf69314820b8aaecb65576978a&role.id=ff2dfc5f80304d9cbd3320d8ceffe2a2 => generated 259 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:58:50.659339 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 452/914] 10.222.0.23 () {58 vars in 1023 bytes} [Thu May 19 11:58:50 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 11:58:50.682482 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34098951-0473-4101-a992-37289887456a None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:50.796671 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232bb4da-6808-4a87-8c65-5857522c0a11 None None] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:50.797486 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232bb4da-6808-4a87-8c65-5857522c0a11 None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:50.797486 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232bb4da-6808-4a87-8c65-5857522c0a11 None None] PATH_INFO: `/` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:50.798675 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 453/915] 10.222.0.23 () {54 vars in 926 bytes} [Thu May 19 11:58:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 11:58:50.807150 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f193ebd3-e177-4a0e-9a75-496422a47acb None None] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:50.807150 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f193ebd3-e177-4a0e-9a75-496422a47acb None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:50.807150 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f193ebd3-e177-4a0e-9a75-496422a47acb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:50.808122 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34098951-0473-4101-a992-37289887456a None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:50.823700 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.auth.core [None req-f193ebd3-e177-4a0e-9a75-496422a47acb None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 11:58:50.825302 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-f193ebd3-e177-4a0e-9a75-496422a47acb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 11:58:50.851047 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d0760a-05f1-43c8-bb58-b08ad894e2ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:50.851890 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d0760a-05f1-43c8-bb58-b08ad894e2ea None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:50.852180 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d0760a-05f1-43c8-bb58-b08ad894e2ea None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:50.852460 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d0760a-05f1-43c8-bb58-b08ad894e2ea None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:50.853414 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d0760a-05f1-43c8-bb58-b08ad894e2ea None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:50.855030 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d0760a-05f1-43c8-bb58-b08ad894e2ea None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:50.871016 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application [None req-57d0760a-05f1-43c8-bb58-b08ad894e2ea None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. May 19 11:58:50.871016 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 11:58:50.871016 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 11:58:50.871016 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 11:58:50.871016 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 11:58:50.871016 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 11:58:50.871016 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 11:58:50.871016 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 11:58:50.871016 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 19 11:58:50.871016 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application May 19 11:58:50.871016 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 11:58:50.871016 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application May 19 11:58:50.871016 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 11:58:50.871016 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 11:58:50.871016 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 11:58:50.871016 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 11:58:50.871016 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 11:58:50.872096 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 11:58:50.872096 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 11:58:50.872096 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 11:58:50.872096 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 11:58:50.872096 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 11:58:50.872096 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 11:58:50.872096 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 11:58:50.872096 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 11:58:50.872096 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 11:58:50.872096 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 11:58:50.872096 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 11:58:50.872096 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 11:58:50.872096 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 11:58:50.872096 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 11:58:50.872096 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 11:58:50.872096 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.get_or_create( May 19 11:58:50.873595 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 11:58:50.873595 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application with Lock( May 19 11:58:50.873595 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 11:58:50.873595 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._enter() May 19 11:58:50.873595 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 11:58:50.873595 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 11:58:50.873595 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 11:58:50.873595 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.creator() May 19 11:58:50.873595 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 11:58:50.873595 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application created_value = creator( May 19 11:58:50.873595 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 11:58:50.873595 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 11:58:50.873595 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 19 11:58:50.873595 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application May 19 11:58:50.873595 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 463/916] 10.222.0.23 () {56 vars in 1114 bytes} [Thu May 19 11:58:50 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 11:58:50.878905 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d0760a-05f1-43c8-bb58-b08ad894e2ea None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:50.880202 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:58:50.912179 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:58:50.923938 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 454/917] 10.222.0.23 () {58 vars in 1023 bytes} [Thu May 19 11:58:50 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 11:58:50.948956 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f193ebd3-e177-4a0e-9a75-496422a47acb None None] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:51.011822 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d0760a-05f1-43c8-bb58-b08ad894e2ea None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:51.061684 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61ff04d-dca1-4511-91fa-6f7c7e8cd30d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:51.062379 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61ff04d-dca1-4511-91fa-6f7c7e8cd30d None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:51.063089 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61ff04d-dca1-4511-91fa-6f7c7e8cd30d None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:51.063384 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61ff04d-dca1-4511-91fa-6f7c7e8cd30d None admin] PATH_INFO: `/v3/services` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:51.064485 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61ff04d-dca1-4511-91fa-6f7c7e8cd30d None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 11:58:51.064871 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61ff04d-dca1-4511-91fa-6f7c7e8cd30d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:51.066279 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61ff04d-dca1-4511-91fa-6f7c7e8cd30d None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:51.071239 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f193ebd3-e177-4a0e-9a75-496422a47acb None None] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:51.073184 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 464/918] 10.222.0.23 () {56 vars in 1114 bytes} [Thu May 19 11:58:50 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:58:51.078439 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61ff04d-dca1-4511-91fa-6f7c7e8cd30d None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:51.112827 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41632fc-5ec1-45d9-abea-0cc6d8c6114e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:51.114080 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41632fc-5ec1-45d9-abea-0cc6d8c6114e None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:51.114458 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41632fc-5ec1-45d9-abea-0cc6d8c6114e None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:51.114736 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41632fc-5ec1-45d9-abea-0cc6d8c6114e None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:51.115731 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41632fc-5ec1-45d9-abea-0cc6d8c6114e None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:51.117853 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41632fc-5ec1-45d9-abea-0cc6d8c6114e None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:51.134042 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application [None req-f41632fc-5ec1-45d9-abea-0cc6d8c6114e None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 11:58:51.134042 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 11:58:51.134042 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 11:58:51.134042 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 11:58:51.134042 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 11:58:51.134042 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 11:58:51.134042 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 11:58:51.134042 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 11:58:51.134042 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 11:58:51.134042 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application May 19 11:58:51.134042 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 11:58:51.134042 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application May 19 11:58:51.134042 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 11:58:51.134042 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 11:58:51.134042 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 11:58:51.134042 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 11:58:51.134042 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 11:58:51.135093 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 11:58:51.135093 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 11:58:51.135093 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 11:58:51.135093 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 11:58:51.135093 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 11:58:51.135093 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 11:58:51.135093 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 11:58:51.135093 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 11:58:51.135093 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 11:58:51.135093 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 11:58:51.135093 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 11:58:51.135093 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 11:58:51.135093 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 11:58:51.135093 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 11:58:51.135093 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 11:58:51.135093 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.get_or_create( May 19 11:58:51.135975 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 11:58:51.135975 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application with Lock( May 19 11:58:51.135975 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 11:58:51.135975 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._enter() May 19 11:58:51.135975 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 11:58:51.135975 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 11:58:51.135975 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 11:58:51.135975 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.creator() May 19 11:58:51.135975 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 11:58:51.135975 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application created_value = creator( May 19 11:58:51.135975 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 11:58:51.135975 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 11:58:51.135975 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 11:58:51.135975 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application May 19 11:58:51.136694 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 455/919] 10.222.0.23 () {56 vars in 1126 bytes} [Thu May 19 11:58:50 2022] GET /identity/v3/services/object-store => generated 93 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 11:58:51.144177 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41632fc-5ec1-45d9-abea-0cc6d8c6114e None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:51.180291 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:58:51.182633 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:58:51.200525 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61ff04d-dca1-4511-91fa-6f7c7e8cd30d None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:51.250488 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8976cc-2c9d-4b9c-a988-f3af7c865dcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:51.252091 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8976cc-2c9d-4b9c-a988-f3af7c865dcc None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:51.252683 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8976cc-2c9d-4b9c-a988-f3af7c865dcc None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:51.253183 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8976cc-2c9d-4b9c-a988-f3af7c865dcc None admin] PATH_INFO: `/v3/services` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:51.254411 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8976cc-2c9d-4b9c-a988-f3af7c865dcc None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 11:58:51.255330 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8976cc-2c9d-4b9c-a988-f3af7c865dcc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:51.257488 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8976cc-2c9d-4b9c-a988-f3af7c865dcc None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:51.265663 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 465/920] 10.222.0.23 () {56 vars in 1114 bytes} [Thu May 19 11:58:51 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:58:51.270741 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8976cc-2c9d-4b9c-a988-f3af7c865dcc None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:51.281731 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41632fc-5ec1-45d9-abea-0cc6d8c6114e None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:51.332215 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef71e3b-96e2-42a3-b8b7-b0a5c33c8c2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:51.332841 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef71e3b-96e2-42a3-b8b7-b0a5c33c8c2d None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:51.332923 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef71e3b-96e2-42a3-b8b7-b0a5c33c8c2d None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:51.333244 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef71e3b-96e2-42a3-b8b7-b0a5c33c8c2d None admin] PATH_INFO: `/v3/services` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:51.333971 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef71e3b-96e2-42a3-b8b7-b0a5c33c8c2d None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 11:58:51.334284 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef71e3b-96e2-42a3-b8b7-b0a5c33c8c2d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:51.335667 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef71e3b-96e2-42a3-b8b7-b0a5c33c8c2d None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:51.346528 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 456/921] 10.222.0.23 () {56 vars in 1122 bytes} [Thu May 19 11:58:51 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:58:51.347774 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef71e3b-96e2-42a3-b8b7-b0a5c33c8c2d None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:51.397996 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8976cc-2c9d-4b9c-a988-f3af7c865dcc None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:51.449430 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ab92ee-ccec-4d2c-83e1-d49d5dcb15ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:51.450352 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ab92ee-ccec-4d2c-83e1-d49d5dcb15ae None admin] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:51.451340 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ab92ee-ccec-4d2c-83e1-d49d5dcb15ae None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:51.452024 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ab92ee-ccec-4d2c-83e1-d49d5dcb15ae None admin] PATH_INFO: `/v3/endpoints` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:51.452787 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ab92ee-ccec-4d2c-83e1-d49d5dcb15ae None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:51.454333 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:58:51.456327 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ab92ee-ccec-4d2c-83e1-d49d5dcb15ae None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:51.460412 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef71e3b-96e2-42a3-b8b7-b0a5c33c8c2d None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:51.496473 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d9183f-63a8-46be-9737-045a6b37cc13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:51.497145 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d9183f-63a8-46be-9737-045a6b37cc13 None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:51.497424 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d9183f-63a8-46be-9737-045a6b37cc13 None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:51.498572 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d9183f-63a8-46be-9737-045a6b37cc13 None admin] PATH_INFO: `/v3/services` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:51.499303 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d9183f-63a8-46be-9737-045a6b37cc13 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 11:58:51.499617 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d9183f-63a8-46be-9737-045a6b37cc13 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:51.501023 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d9183f-63a8-46be-9737-045a6b37cc13 None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:51.506609 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 457/922] 10.222.0.23 () {56 vars in 1122 bytes} [Thu May 19 11:58:51 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:58:51.512297 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d9183f-63a8-46be-9737-045a6b37cc13 None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:51.538549 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:58:51.548200 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 466/923] 10.222.0.23 () {60 vars in 1144 bytes} [Thu May 19 11:58:51 2022] POST /identity/v3/endpoints => generated 346 bytes in 278 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 11:58:51.632306 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d9183f-63a8-46be-9737-045a6b37cc13 None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:51.678916 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eacabd4-0162-4286-b858-8a87e0854d7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:51.679657 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eacabd4-0162-4286-b858-8a87e0854d7b None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:51.679981 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eacabd4-0162-4286-b858-8a87e0854d7b None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:51.680263 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eacabd4-0162-4286-b858-8a87e0854d7b None admin] PATH_INFO: `/v3/endpoints` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:51.681191 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eacabd4-0162-4286-b858-8a87e0854d7b None admin] RBAC: Adding query filter params (interface=public, service_id=0ba01aa9ffd04ed18443612845ad5952, region_id=RegionOne) {{(pid=80539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 11:58:51.681515 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eacabd4-0162-4286-b858-8a87e0854d7b None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:51.682946 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eacabd4-0162-4286-b858-8a87e0854d7b None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:51.694780 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 458/924] 10.222.0.23 () {56 vars in 1251 bytes} [Thu May 19 11:58:51 2022] GET /identity/v3/endpoints?service_id=0ba01aa9ffd04ed18443612845ad5952&interface=public®ion_id=RegionOne => generated 194 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 11:58:51.696632 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ab92ee-ccec-4d2c-83e1-d49d5dcb15ae None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:51.823439 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ab92ee-ccec-4d2c-83e1-d49d5dcb15ae None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:51.864727 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66e7de1-7f24-4258-a873-00571f13910a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:51.865487 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66e7de1-7f24-4258-a873-00571f13910a None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:51.865776 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66e7de1-7f24-4258-a873-00571f13910a None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:51.866061 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66e7de1-7f24-4258-a873-00571f13910a None admin] PATH_INFO: `/v3/services` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:51.866797 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66e7de1-7f24-4258-a873-00571f13910a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:51.868242 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66e7de1-7f24-4258-a873-00571f13910a None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:51.879383 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 467/925] 10.222.0.23 () {56 vars in 1087 bytes} [Thu May 19 11:58:51 2022] GET /identity/v3/services => generated 1497 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 11:58:51.952400 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01596f36-ceb7-470b-9310-922c767f3f10 None None] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:51.952693 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01596f36-ceb7-470b-9310-922c767f3f10 None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:51.952976 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01596f36-ceb7-470b-9310-922c767f3f10 None None] PATH_INFO: `/` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:51.954713 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 459/926] 10.222.0.23 () {54 vars in 926 bytes} [Thu May 19 11:58:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 11:58:51.960555 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579564e8-5afe-41c6-b5b3-0d0defa8f0dd None None] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:51.960862 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579564e8-5afe-41c6-b5b3-0d0defa8f0dd None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:51.961133 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579564e8-5afe-41c6-b5b3-0d0defa8f0dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:51.980250 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.auth.core [None req-579564e8-5afe-41c6-b5b3-0d0defa8f0dd None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 11:58:51.981566 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-579564e8-5afe-41c6-b5b3-0d0defa8f0dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 11:58:52.054981 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:58:52.064760 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 468/927] 10.222.0.23 () {58 vars in 1023 bytes} [Thu May 19 11:58:51 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 11:58:52.088036 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01596f36-ceb7-470b-9310-922c767f3f10 None None] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:52.114206 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:58:52.202386 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01596f36-ceb7-470b-9310-922c767f3f10 None None] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:52.239336 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b004cae-d636-49b5-a8bd-14680e26e819 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:52.240559 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b004cae-d636-49b5-a8bd-14680e26e819 None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:52.241052 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b004cae-d636-49b5-a8bd-14680e26e819 None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:52.241532 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b004cae-d636-49b5-a8bd-14680e26e819 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:52.242461 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b004cae-d636-49b5-a8bd-14680e26e819 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:52.244079 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b004cae-d636-49b5-a8bd-14680e26e819 None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:52.257233 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application [None req-0b004cae-d636-49b5-a8bd-14680e26e819 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 19 11:58:52.257233 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 11:58:52.257233 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 11:58:52.257233 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 11:58:52.257233 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 11:58:52.257233 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 11:58:52.257233 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 11:58:52.257233 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 11:58:52.257233 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 19 11:58:52.257233 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application May 19 11:58:52.257233 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 11:58:52.257233 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application May 19 11:58:52.257233 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 11:58:52.257233 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 11:58:52.257233 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 11:58:52.257233 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 11:58:52.258188 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 11:58:52.258188 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 11:58:52.258188 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 11:58:52.258188 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 11:58:52.258188 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 11:58:52.258188 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 11:58:52.258188 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 11:58:52.258188 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 11:58:52.258188 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 11:58:52.258188 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 11:58:52.258188 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 11:58:52.258188 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 11:58:52.258188 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 11:58:52.258188 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 11:58:52.258188 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 11:58:52.258188 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 11:58:52.259008 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.get_or_create( May 19 11:58:52.259008 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 11:58:52.259008 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application with Lock( May 19 11:58:52.259008 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 11:58:52.259008 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._enter() May 19 11:58:52.259008 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 11:58:52.259008 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 11:58:52.259008 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 11:58:52.259008 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.creator() May 19 11:58:52.259008 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 11:58:52.259008 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application created_value = creator( May 19 11:58:52.259008 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 11:58:52.259008 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 11:58:52.259008 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 19 11:58:52.259008 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application May 19 11:58:52.261927 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 460/928] 10.222.0.23 () {56 vars in 1132 bytes} [Thu May 19 11:58:52 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 11:58:52.271806 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579564e8-5afe-41c6-b5b3-0d0defa8f0dd None None] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:52.379028 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579564e8-5afe-41c6-b5b3-0d0defa8f0dd None None] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:52.416022 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb12fe5-d125-41d6-a825-23c5057c8b8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:52.416701 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb12fe5-d125-41d6-a825-23c5057c8b8f None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:52.416992 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb12fe5-d125-41d6-a825-23c5057c8b8f None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:52.417422 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb12fe5-d125-41d6-a825-23c5057c8b8f None admin] PATH_INFO: `/v3/services` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:52.418059 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb12fe5-d125-41d6-a825-23c5057c8b8f None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 11:58:52.418373 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb12fe5-d125-41d6-a825-23c5057c8b8f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:52.420181 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb12fe5-d125-41d6-a825-23c5057c8b8f None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:52.425851 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 469/929] 10.222.0.23 () {56 vars in 1126 bytes} [Thu May 19 11:58:52 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:58:52.431095 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b004cae-d636-49b5-a8bd-14680e26e819 None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:52.527636 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:58:52.544263 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b004cae-d636-49b5-a8bd-14680e26e819 None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:52.580310 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611d0fe0-1320-4d68-9dfa-6a18fea6c261 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:52.580992 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611d0fe0-1320-4d68-9dfa-6a18fea6c261 None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:52.581272 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611d0fe0-1320-4d68-9dfa-6a18fea6c261 None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:52.581546 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611d0fe0-1320-4d68-9dfa-6a18fea6c261 None admin] PATH_INFO: `/v3/services` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:52.582257 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611d0fe0-1320-4d68-9dfa-6a18fea6c261 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 11:58:52.582564 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611d0fe0-1320-4d68-9dfa-6a18fea6c261 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:52.583945 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611d0fe0-1320-4d68-9dfa-6a18fea6c261 None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:52.589367 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 461/930] 10.222.0.23 () {56 vars in 1126 bytes} [Thu May 19 11:58:52 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:58:52.595715 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb12fe5-d125-41d6-a825-23c5057c8b8f None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:52.659384 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:58:52.707517 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb12fe5-d125-41d6-a825-23c5057c8b8f None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:52.743204 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9354963e-ea0a-4b8b-9e55-a65a33273c8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:52.743967 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9354963e-ea0a-4b8b-9e55-a65a33273c8f None admin] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:52.744262 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9354963e-ea0a-4b8b-9e55-a65a33273c8f None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:52.744547 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9354963e-ea0a-4b8b-9e55-a65a33273c8f None admin] PATH_INFO: `/v3/endpoints` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:52.745357 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9354963e-ea0a-4b8b-9e55-a65a33273c8f None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:52.746783 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9354963e-ea0a-4b8b-9e55-a65a33273c8f None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:52.836091 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 470/931] 10.222.0.23 () {60 vars in 1144 bytes} [Thu May 19 11:58:52 2022] POST /identity/v3/endpoints => generated 347 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 11:58:52.981425 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f5415f-6f9a-4ff2-931c-7630ee1ef8c8 None None] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:52.981798 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f5415f-6f9a-4ff2-931c-7630ee1ef8c8 None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:52.982138 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f5415f-6f9a-4ff2-931c-7630ee1ef8c8 None None] PATH_INFO: `/` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:52.983826 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 462/932] 10.222.0.23 () {54 vars in 926 bytes} [Thu May 19 11:58:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 11:58:52.989496 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd92acb-c2da-4dba-bc4c-7c22136cd9f7 None None] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:52.989816 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd92acb-c2da-4dba-bc4c-7c22136cd9f7 None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:52.990058 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd92acb-c2da-4dba-bc4c-7c22136cd9f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:53.008129 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.auth.core [None req-6cd92acb-c2da-4dba-bc4c-7c22136cd9f7 None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 11:58:53.009347 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-6cd92acb-c2da-4dba-bc4c-7c22136cd9f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 11:58:53.090497 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 471/933] 10.222.0.23 () {58 vars in 1023 bytes} [Thu May 19 11:58:52 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 11:58:53.115770 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f5415f-6f9a-4ff2-931c-7630ee1ef8c8 None None] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:53.231488 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f5415f-6f9a-4ff2-931c-7630ee1ef8c8 None None] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:53.268880 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7fbb5f-9dd2-4750-ac32-f5d854a2e2e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:53.269889 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7fbb5f-9dd2-4750-ac32-f5d854a2e2e4 None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:53.270175 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7fbb5f-9dd2-4750-ac32-f5d854a2e2e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:53.270449 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7fbb5f-9dd2-4750-ac32-f5d854a2e2e4 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:53.282980 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7fbb5f-9dd2-4750-ac32-f5d854a2e2e4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 11:58:53.291319 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application [None req-8c7fbb5f-9dd2-4750-ac32-f5d854a2e2e4 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 19 11:58:53.291319 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 11:58:53.291319 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 11:58:53.291319 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 11:58:53.291319 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 11:58:53.291319 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 11:58:53.291319 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 11:58:53.291319 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 11:58:53.291319 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 11:58:53.291319 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 11:58:53.291319 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 11:58:53.291319 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 11:58:53.291319 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 11:58:53.291319 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 11:58:53.291319 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 11:58:53.291319 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 11:58:53.292370 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 11:58:53.292370 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 11:58:53.292370 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 11:58:53.292370 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 11:58:53.292370 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 11:58:53.292370 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 11:58:53.292370 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 11:58:53.292370 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 11:58:53.292370 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 11:58:53.292370 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.get_or_create( May 19 11:58:53.292370 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 11:58:53.292370 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application with Lock( May 19 11:58:53.292370 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 11:58:53.292370 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._enter() May 19 11:58:53.292370 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 11:58:53.292370 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 11:58:53.293224 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 11:58:53.293224 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.creator() May 19 11:58:53.293224 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 11:58:53.293224 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application created_value = creator( May 19 11:58:53.293224 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 11:58:53.293224 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 11:58:53.293224 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 11:58:53.293224 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 11:58:53.293224 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 11:58:53.293224 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 11:58:53.293224 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 19 11:58:53.293224 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application May 19 11:58:53.293224 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 463/934] 10.222.0.23 () {56 vars in 1120 bytes} [Thu May 19 11:58:53 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 11:58:53.299045 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd92acb-c2da-4dba-bc4c-7c22136cd9f7 None None] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:53.343450 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:58:53.406462 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd92acb-c2da-4dba-bc4c-7c22136cd9f7 None None] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:53.446081 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5408b1a-9790-4ae0-bc53-a289669da904 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:53.447279 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5408b1a-9790-4ae0-bc53-a289669da904 None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:53.447574 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5408b1a-9790-4ae0-bc53-a289669da904 None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:53.447866 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5408b1a-9790-4ae0-bc53-a289669da904 None admin] PATH_INFO: `/v3/roles` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:53.448616 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5408b1a-9790-4ae0-bc53-a289669da904 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 11:58:53.448923 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5408b1a-9790-4ae0-bc53-a289669da904 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:53.450386 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5408b1a-9790-4ae0-bc53-a289669da904 None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:53.459768 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 472/935] 10.222.0.23 () {56 vars in 1115 bytes} [Thu May 19 11:58:53 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:58:53.465392 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7fbb5f-9dd2-4750-ac32-f5d854a2e2e4 None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:53.476958 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:58:53.580175 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7fbb5f-9dd2-4750-ac32-f5d854a2e2e4 None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:53.618079 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b634799-c415-4ca2-b86b-8e8444d025c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:53.618798 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b634799-c415-4ca2-b86b-8e8444d025c5 None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:53.619064 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b634799-c415-4ca2-b86b-8e8444d025c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:53.619333 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b634799-c415-4ca2-b86b-8e8444d025c5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:53.630963 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b634799-c415-4ca2-b86b-8e8444d025c5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:53.632455 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b634799-c415-4ca2-b86b-8e8444d025c5 None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:53.651182 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application [None req-2b634799-c415-4ca2-b86b-8e8444d025c5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 11:58:53.651182 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 11:58:53.651182 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 11:58:53.651182 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 11:58:53.651182 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 11:58:53.651182 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 11:58:53.651182 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 11:58:53.651182 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 11:58:53.651182 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 11:58:53.651182 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 11:58:53.651182 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 11:58:53.651182 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 11:58:53.651182 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 11:58:53.651182 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 11:58:53.651182 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 11:58:53.653380 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 11:58:53.653380 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 11:58:53.653380 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 11:58:53.653380 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 11:58:53.653380 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 11:58:53.653380 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 11:58:53.653380 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.get_or_create( May 19 11:58:53.653380 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 11:58:53.653380 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application with Lock( May 19 11:58:53.653380 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 11:58:53.653380 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._enter() May 19 11:58:53.653380 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 11:58:53.653380 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 11:58:53.653380 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 11:58:53.653380 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.creator() May 19 11:58:53.653380 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 11:58:53.653380 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application created_value = creator( May 19 11:58:53.655727 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 11:58:53.655727 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 11:58:53.655727 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 11:58:53.655727 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application May 19 11:58:53.655727 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 464/936] 10.222.0.23 () {56 vars in 1108 bytes} [Thu May 19 11:58:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 11:58:53.661161 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5408b1a-9790-4ae0-bc53-a289669da904 None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:53.771152 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5408b1a-9790-4ae0-bc53-a289669da904 None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:53.772975 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:58:53.814316 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216f6f15-1789-4835-b5ab-be0fd8c58a8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:53.815269 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216f6f15-1789-4835-b5ab-be0fd8c58a8e None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:53.815706 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216f6f15-1789-4835-b5ab-be0fd8c58a8e None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:53.816023 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216f6f15-1789-4835-b5ab-be0fd8c58a8e None admin] PATH_INFO: `/v3/domains` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:53.816761 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216f6f15-1789-4835-b5ab-be0fd8c58a8e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 11:58:53.817071 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216f6f15-1789-4835-b5ab-be0fd8c58a8e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:53.818465 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216f6f15-1789-4835-b5ab-be0fd8c58a8e None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:53.838064 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 473/937] 10.222.0.23 () {56 vars in 1109 bytes} [Thu May 19 11:58:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 11:58:53.842318 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b634799-c415-4ca2-b86b-8e8444d025c5 None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:53.910029 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:58:53.962934 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b634799-c415-4ca2-b86b-8e8444d025c5 None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:53.999089 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffab62da-4e9b-459d-ad7d-5a19e71cec75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:53.999946 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffab62da-4e9b-459d-ad7d-5a19e71cec75 None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:54.000233 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffab62da-4e9b-459d-ad7d-5a19e71cec75 None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:54.000505 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffab62da-4e9b-459d-ad7d-5a19e71cec75 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:54.008601 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffab62da-4e9b-459d-ad7d-5a19e71cec75 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:54.010068 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffab62da-4e9b-459d-ad7d-5a19e71cec75 None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:54.026684 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application [None req-ffab62da-4e9b-459d-ad7d-5a19e71cec75 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 19 11:58:54.026684 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 11:58:54.026684 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 11:58:54.026684 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 11:58:54.026684 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 11:58:54.026684 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 11:58:54.026684 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 11:58:54.026684 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 11:58:54.026684 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 11:58:54.026684 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 11:58:54.026684 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 11:58:54.026684 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 11:58:54.026684 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 11:58:54.026684 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 11:58:54.026684 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 11:58:54.026684 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 11:58:54.027681 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 11:58:54.027681 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 11:58:54.027681 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 11:58:54.027681 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 11:58:54.027681 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 11:58:54.027681 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 11:58:54.027681 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 11:58:54.027681 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 11:58:54.027681 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 11:58:54.027681 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.get_or_create( May 19 11:58:54.027681 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 11:58:54.027681 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application with Lock( May 19 11:58:54.027681 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 11:58:54.027681 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._enter() May 19 11:58:54.027681 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 11:58:54.027681 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 11:58:54.027681 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 11:58:54.027681 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.creator() May 19 11:58:54.028659 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 11:58:54.028659 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application created_value = creator( May 19 11:58:54.028659 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 11:58:54.028659 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 11:58:54.028659 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 11:58:54.028659 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 11:58:54.028659 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 11:58:54.028659 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 11:58:54.028659 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 19 11:58:54.028659 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application May 19 11:58:54.028659 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 465/938] 10.222.0.23 () {56 vars in 1117 bytes} [Thu May 19 11:58:53 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 11:58:54.033576 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2f431d-28c5-4214-a111-77f714a81362 None None] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:54.033872 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2f431d-28c5-4214-a111-77f714a81362 None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:54.034153 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2f431d-28c5-4214-a111-77f714a81362 None None] PATH_INFO: `/` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:54.035921 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 474/939] 10.222.0.23 () {54 vars in 926 bytes} [Thu May 19 11:58:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 11:58:54.038512 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffab62da-4e9b-459d-ad7d-5a19e71cec75 None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:54.041697 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738dbb6d-248b-4da3-9c7d-ee36cb96f711 None None] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:54.042221 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738dbb6d-248b-4da3-9c7d-ee36cb96f711 None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:54.042714 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738dbb6d-248b-4da3-9c7d-ee36cb96f711 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:54.060656 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.auth.core [None req-738dbb6d-248b-4da3-9c7d-ee36cb96f711 None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 11:58:54.062374 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-738dbb6d-248b-4da3-9c7d-ee36cb96f711 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 11:58:54.148341 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 475/940] 10.222.0.23 () {58 vars in 1023 bytes} [Thu May 19 11:58:54 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 11:58:54.164367 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffab62da-4e9b-459d-ad7d-5a19e71cec75 None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:54.170759 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738dbb6d-248b-4da3-9c7d-ee36cb96f711 None None] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:54.189054 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:58:54.207502 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d78808-6ae2-4c94-b587-90d34e09a95f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:54.208213 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d78808-6ae2-4c94-b587-90d34e09a95f None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:54.208494 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d78808-6ae2-4c94-b587-90d34e09a95f None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:54.208772 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d78808-6ae2-4c94-b587-90d34e09a95f None admin] PATH_INFO: `/v3/users` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:54.209790 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d78808-6ae2-4c94-b587-90d34e09a95f None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 11:58:54.210105 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d78808-6ae2-4c94-b587-90d34e09a95f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:54.212415 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d78808-6ae2-4c94-b587-90d34e09a95f None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:54.238776 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 466/941] 10.222.0.23 () {56 vars in 1149 bytes} [Thu May 19 11:58:54 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:58:54.243504 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d78808-6ae2-4c94-b587-90d34e09a95f None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:54.283739 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:58:54.293574 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738dbb6d-248b-4da3-9c7d-ee36cb96f711 None None] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:54.331084 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10a9aa9-04e3-430e-a508-0d049a6013f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:54.331815 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10a9aa9-04e3-430e-a508-0d049a6013f2 None admin] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:54.332126 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10a9aa9-04e3-430e-a508-0d049a6013f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:54.332402 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10a9aa9-04e3-430e-a508-0d049a6013f2 None admin] PATH_INFO: `/v3/projects` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:54.333109 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10a9aa9-04e3-430e-a508-0d049a6013f2 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:54.334592 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10a9aa9-04e3-430e-a508-0d049a6013f2 None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:54.336260 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: WARNING keystone.server.flask.common [None req-d10a9aa9-04e3-430e-a508-0d049a6013f2 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. May 19 11:58:54.350040 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d78808-6ae2-4c94-b587-90d34e09a95f None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:54.376602 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.sql.core [None req-d10a9aa9-04e3-430e-a508-0d049a6013f2 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") May 19 11:58:54.376602 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] May 19 11:58:54.376602 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [parameters: {'id': '25f0d0d1349c4494b88fa95c5a47bb4b', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] May 19 11:58:54.376602 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80540) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} May 19 11:58:54.384239 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application [None req-d10a9aa9-04e3-430e-a508-0d049a6013f2 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 25f0d0d1349c4494b88fa95c5a47bb4b.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 25f0d0d1349c4494b88fa95c5a47bb4b. May 19 11:58:54.384239 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 11:58:54.384239 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 19 11:58:54.384239 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application self.dialect.do_execute( May 19 11:58:54.384239 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 19 11:58:54.384239 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 19 11:58:54.384239 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 19 11:58:54.384239 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application result = self._query(query) May 19 11:58:54.384239 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 19 11:58:54.384239 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application conn.query(q) May 19 11:58:54.384239 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 19 11:58:54.384239 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 19 11:58:54.384239 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 19 11:58:54.385012 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application result.read() May 19 11:58:54.385012 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 19 11:58:54.385012 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 19 11:58:54.385012 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 19 11:58:54.385012 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application packet.raise_for_error() May 19 11:58:54.385012 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 19 11:58:54.385012 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 19 11:58:54.385012 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 19 11:58:54.385012 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 19 11:58:54.385012 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") May 19 11:58:54.385012 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application May 19 11:58:54.385012 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: May 19 11:58:54.385012 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application May 19 11:58:54.385012 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 11:58:54.385012 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper May 19 11:58:54.385012 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return method(*args, **kwargs) May 19 11:58:54.385012 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project May 19 11:58:54.385012 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return project_ref.to_dict() May 19 11:58:54.386219 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 19 11:58:54.386219 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application next(self.gen) May 19 11:58:54.386219 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope May 19 11:58:54.386219 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application yield resource May 19 11:58:54.386219 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 19 11:58:54.386219 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application next(self.gen) May 19 11:58:54.386219 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session May 19 11:58:54.386219 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application self.session.rollback() May 19 11:58:54.386219 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ May 19 11:58:54.386219 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application self.force_reraise() May 19 11:58:54.386219 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise May 19 11:58:54.386219 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application raise self.value May 19 11:58:54.386219 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session May 19 11:58:54.386219 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) May 19 11:58:54.386219 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction May 19 11:58:54.386219 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application session.commit() May 19 11:58:54.386219 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1435, in commit May 19 11:58:54.387401 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) May 19 11:58:54.387401 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit May 19 11:58:54.387401 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application self._prepare_impl() May 19 11:58:54.387401 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl May 19 11:58:54.387401 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application self.session.flush() May 19 11:58:54.387401 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3367, in flush May 19 11:58:54.387401 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application self._flush(objects) May 19 11:58:54.387401 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3507, in _flush May 19 11:58:54.387401 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) May 19 11:58:54.387401 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ May 19 11:58:54.387401 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application compat.raise_( May 19 11:58:54.387401 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 19 11:58:54.387401 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application raise exception May 19 11:58:54.387401 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3467, in _flush May 19 11:58:54.387401 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application flush_context.execute() May 19 11:58:54.387401 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute May 19 11:58:54.387401 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application rec.execute(self) May 19 11:58:54.388388 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute May 19 11:58:54.388388 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( May 19 11:58:54.388388 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj May 19 11:58:54.388388 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application _emit_insert_statements( May 19 11:58:54.388388 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements May 19 11:58:54.388388 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application c = connection._execute_20( May 19 11:58:54.388388 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 May 19 11:58:54.388388 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) May 19 11:58:54.388388 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection May 19 11:58:54.388388 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return connection._execute_clauseelement( May 19 11:58:54.388388 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement May 19 11:58:54.388388 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application ret = self._execute_context( May 19 11:58:54.388388 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context May 19 11:58:54.388388 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application self._handle_dbapi_exception( May 19 11:58:54.388388 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception May 19 11:58:54.390233 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) May 19 11:58:54.390233 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 19 11:58:54.390233 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application raise exception May 19 11:58:54.390233 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 19 11:58:54.390233 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application self.dialect.do_execute( May 19 11:58:54.390233 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 19 11:58:54.390233 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 19 11:58:54.390233 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 19 11:58:54.390233 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application result = self._query(query) May 19 11:58:54.390233 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 19 11:58:54.390233 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application conn.query(q) May 19 11:58:54.390233 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 19 11:58:54.390233 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 19 11:58:54.390233 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 19 11:58:54.390233 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application result.read() May 19 11:58:54.390233 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 19 11:58:54.390233 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 19 11:58:54.392411 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 19 11:58:54.392411 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application packet.raise_for_error() May 19 11:58:54.392411 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 19 11:58:54.392411 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 19 11:58:54.392411 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 19 11:58:54.392411 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 19 11:58:54.392411 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") May 19 11:58:54.392411 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] May 19 11:58:54.392411 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application [parameters: {'id': '25f0d0d1349c4494b88fa95c5a47bb4b', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] May 19 11:58:54.392411 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) May 19 11:58:54.392411 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application May 19 11:58:54.392411 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 11:58:54.392411 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application May 19 11:58:54.392411 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 11:58:54.392411 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project May 19 11:58:54.396864 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) May 19 11:58:54.396864 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper May 19 11:58:54.396864 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, May 19 11:58:54.396864 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. May 19 11:58:54.396864 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application May 19 11:58:54.396864 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 11:58:54.396864 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application May 19 11:58:54.396864 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 11:58:54.396864 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 11:58:54.396864 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 11:58:54.396864 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 11:58:54.396864 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 11:58:54.396864 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 11:58:54.396864 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 11:58:54.396864 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 11:58:54.396864 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 11:58:54.396864 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 11:58:54.397773 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 11:58:54.397773 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post May 19 11:58:54.397773 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( May 19 11:58:54.397773 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 11:58:54.397773 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 11:58:54.397773 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project May 19 11:58:54.397773 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application raise exception.Conflict( May 19 11:58:54.397773 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 25f0d0d1349c4494b88fa95c5a47bb4b. May 19 11:58:54.397773 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application May 19 11:58:54.397773 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 476/942] 10.222.0.23 () {60 vars in 1140 bytes} [Thu May 19 11:58:54 2022] POST /identity/v3/projects => generated 262 bytes in 216 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) May 19 11:58:54.397773 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f177f2c4-4fc8-40b4-816e-4b55a589f6ce None None] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:54.397773 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f177f2c4-4fc8-40b4-816e-4b55a589f6ce None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:54.398415 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f177f2c4-4fc8-40b4-816e-4b55a589f6ce None None] PATH_INFO: `/` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:54.398415 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 477/943] 10.222.0.23 () {54 vars in 926 bytes} [Thu May 19 11:58:54 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 11:58:54.398544 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776c8b51-85bb-455e-97e7-32563cdd7338 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:54.398632 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f177f2c4-4fc8-40b4-816e-4b55a589f6ce None None] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:54.398701 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776c8b51-85bb-455e-97e7-32563cdd7338 None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:54.398701 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776c8b51-85bb-455e-97e7-32563cdd7338 None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:54.398701 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776c8b51-85bb-455e-97e7-32563cdd7338 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:54.412223 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776c8b51-85bb-455e-97e7-32563cdd7338 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:54.413709 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776c8b51-85bb-455e-97e7-32563cdd7338 None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:54.432615 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application [None req-776c8b51-85bb-455e-97e7-32563cdd7338 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 11:58:54.432615 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 11:58:54.432615 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 11:58:54.432615 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 11:58:54.432615 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 11:58:54.432615 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 11:58:54.432615 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 11:58:54.432615 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 11:58:54.432615 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 11:58:54.432615 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 11:58:54.432615 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 11:58:54.432615 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 11:58:54.432615 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 11:58:54.432615 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 11:58:54.432615 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 11:58:54.433516 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 11:58:54.433516 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 11:58:54.433516 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 11:58:54.433516 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 11:58:54.433516 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 11:58:54.433516 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 11:58:54.433516 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.get_or_create( May 19 11:58:54.433516 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 11:58:54.433516 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application with Lock( May 19 11:58:54.433516 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 11:58:54.433516 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._enter() May 19 11:58:54.433516 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 11:58:54.433516 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 11:58:54.433516 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 11:58:54.433516 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.creator() May 19 11:58:54.433516 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 11:58:54.433516 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application created_value = creator( May 19 11:58:54.434408 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 11:58:54.434408 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 11:58:54.434408 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 11:58:54.434408 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application May 19 11:58:54.434408 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 467/944] 10.222.0.23 () {56 vars in 1108 bytes} [Thu May 19 11:58:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 11:58:54.436088 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479ca9b9-6afb-4357-b703-b1191bce465b None None] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:54.436088 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479ca9b9-6afb-4357-b703-b1191bce465b None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:54.436380 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479ca9b9-6afb-4357-b703-b1191bce465b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:54.454810 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.auth.core [None req-479ca9b9-6afb-4357-b703-b1191bce465b None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 11:58:54.456035 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-479ca9b9-6afb-4357-b703-b1191bce465b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 11:58:54.481723 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:58:54.517187 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f177f2c4-4fc8-40b4-816e-4b55a589f6ce None None] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:54.531844 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 468/945] 10.222.0.23 () {58 vars in 1023 bytes} [Thu May 19 11:58:54 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 11:58:54.531844 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479ca9b9-6afb-4357-b703-b1191bce465b None None] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:54.566454 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f27c32a-37ff-4578-8a8d-18d63f21b3a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:54.567249 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f27c32a-37ff-4578-8a8d-18d63f21b3a1 None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:54.567537 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f27c32a-37ff-4578-8a8d-18d63f21b3a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:54.567860 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f27c32a-37ff-4578-8a8d-18d63f21b3a1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:54.575525 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f27c32a-37ff-4578-8a8d-18d63f21b3a1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:54.577950 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f27c32a-37ff-4578-8a8d-18d63f21b3a1 None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:54.593255 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application [None req-4f27c32a-37ff-4578-8a8d-18d63f21b3a1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 11:58:54.593255 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 11:58:54.593255 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 11:58:54.593255 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 11:58:54.593255 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 11:58:54.593255 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 11:58:54.593255 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 11:58:54.593255 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 11:58:54.593255 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 11:58:54.593255 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 11:58:54.593255 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 11:58:54.593255 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 11:58:54.593255 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 11:58:54.593255 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 11:58:54.593255 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 11:58:54.594365 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 11:58:54.594365 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 11:58:54.594365 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 11:58:54.594365 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 11:58:54.594365 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 11:58:54.594365 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 11:58:54.594365 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.get_or_create( May 19 11:58:54.594365 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 11:58:54.594365 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application with Lock( May 19 11:58:54.594365 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 11:58:54.594365 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._enter() May 19 11:58:54.594365 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 11:58:54.594365 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 11:58:54.594365 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 11:58:54.594365 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.creator() May 19 11:58:54.594365 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 11:58:54.594365 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application created_value = creator( May 19 11:58:54.595454 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 11:58:54.595454 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 11:58:54.595454 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 11:58:54.595454 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 11:58:54.595454 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 11:58:54.595454 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 11:58:54.595454 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 11:58:54.595454 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application May 19 11:58:54.595454 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 478/946] 10.222.0.23 () {56 vars in 1111 bytes} [Thu May 19 11:58:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 11:58:54.595454 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f27c32a-37ff-4578-8a8d-18d63f21b3a1 None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:54.613908 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:58:54.656195 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479ca9b9-6afb-4357-b703-b1191bce465b None None] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:54.694804 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c51233-8c8f-43d2-87bf-38a9d7f6296c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:54.695392 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c51233-8c8f-43d2-87bf-38a9d7f6296c None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:54.695671 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c51233-8c8f-43d2-87bf-38a9d7f6296c None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:54.696223 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c51233-8c8f-43d2-87bf-38a9d7f6296c None admin] PATH_INFO: `/v3/domains` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:54.696979 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c51233-8c8f-43d2-87bf-38a9d7f6296c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 11:58:54.697286 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c51233-8c8f-43d2-87bf-38a9d7f6296c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:54.698652 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c51233-8c8f-43d2-87bf-38a9d7f6296c None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:54.715973 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 469/947] 10.222.0.23 () {56 vars in 1109 bytes} [Thu May 19 11:58:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:58:54.715973 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c51233-8c8f-43d2-87bf-38a9d7f6296c None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:54.721115 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f27c32a-37ff-4578-8a8d-18d63f21b3a1 None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:54.737169 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:58:54.758910 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b98021-995e-4970-86d4-a9277f8cbf02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:54.759694 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b98021-995e-4970-86d4-a9277f8cbf02 None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:54.760152 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b98021-995e-4970-86d4-a9277f8cbf02 None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:54.760325 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b98021-995e-4970-86d4-a9277f8cbf02 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:54.760938 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b98021-995e-4970-86d4-a9277f8cbf02 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:54.762338 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b98021-995e-4970-86d4-a9277f8cbf02 None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:54.775961 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application [None req-13b98021-995e-4970-86d4-a9277f8cbf02 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 11:58:54.775961 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 11:58:54.775961 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 11:58:54.775961 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 11:58:54.775961 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 11:58:54.775961 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 11:58:54.775961 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 11:58:54.775961 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 11:58:54.775961 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 11:58:54.775961 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application May 19 11:58:54.775961 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 11:58:54.775961 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application May 19 11:58:54.775961 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 11:58:54.775961 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 11:58:54.775961 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 11:58:54.775961 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 11:58:54.775961 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 11:58:54.777002 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 11:58:54.777002 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 11:58:54.777002 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 11:58:54.777002 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 11:58:54.777002 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 11:58:54.777002 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 11:58:54.777002 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 11:58:54.777002 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 11:58:54.777002 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 11:58:54.777002 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 11:58:54.777002 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 11:58:54.777002 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 11:58:54.777002 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 11:58:54.777002 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 11:58:54.777002 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 11:58:54.777002 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.get_or_create( May 19 11:58:54.777812 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 11:58:54.777812 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application with Lock( May 19 11:58:54.777812 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 11:58:54.777812 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._enter() May 19 11:58:54.777812 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 11:58:54.777812 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 11:58:54.777812 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 11:58:54.777812 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.creator() May 19 11:58:54.777812 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 11:58:54.777812 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application created_value = creator( May 19 11:58:54.777812 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 11:58:54.777812 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 11:58:54.777812 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 11:58:54.777812 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application May 19 11:58:54.777812 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 479/948] 10.222.0.23 () {56 vars in 1126 bytes} [Thu May 19 11:58:54 2022] GET /identity/v3/services/object-store => generated 93 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 11:58:54.779652 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b98021-995e-4970-86d4-a9277f8cbf02 None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:54.827937 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c51233-8c8f-43d2-87bf-38a9d7f6296c None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:54.856911 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:58:54.864911 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8950a038-4cc2-481c-9656-ffd5e45eafca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:54.865576 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8950a038-4cc2-481c-9656-ffd5e45eafca None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:54.865859 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8950a038-4cc2-481c-9656-ffd5e45eafca None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:54.866135 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8950a038-4cc2-481c-9656-ffd5e45eafca None admin] PATH_INFO: `/v3/projects` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:54.866890 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8950a038-4cc2-481c-9656-ffd5e45eafca None admin] RBAC: Adding query filter params (name=service) {{(pid=80539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 11:58:54.867476 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8950a038-4cc2-481c-9656-ffd5e45eafca None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:54.868668 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8950a038-4cc2-481c-9656-ffd5e45eafca None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:54.887451 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 470/949] 10.222.0.23 () {56 vars in 1112 bytes} [Thu May 19 11:58:54 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:58:54.889165 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8950a038-4cc2-481c-9656-ffd5e45eafca None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:54.901372 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b98021-995e-4970-86d4-a9277f8cbf02 None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:54.962939 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca296983-3ebd-467e-98dd-a538d10ebe40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:54.964905 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca296983-3ebd-467e-98dd-a538d10ebe40 None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:54.965484 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca296983-3ebd-467e-98dd-a538d10ebe40 None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:54.965983 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca296983-3ebd-467e-98dd-a538d10ebe40 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:54.977860 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca296983-3ebd-467e-98dd-a538d10ebe40 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:54.980991 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca296983-3ebd-467e-98dd-a538d10ebe40 None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:55.001894 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:58:55.005130 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application [None req-ca296983-3ebd-467e-98dd-a538d10ebe40 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 11:58:55.005130 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 11:58:55.005130 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 11:58:55.005130 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 11:58:55.005130 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 11:58:55.005130 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 11:58:55.005130 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 11:58:55.005130 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 11:58:55.005130 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 11:58:55.005130 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 11:58:55.005130 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 11:58:55.005130 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 11:58:55.005130 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 11:58:55.005130 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 11:58:55.005130 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 11:58:55.006111 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 11:58:55.006111 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 11:58:55.006111 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 11:58:55.006111 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 11:58:55.006111 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 11:58:55.006111 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 11:58:55.006111 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.get_or_create( May 19 11:58:55.006111 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 11:58:55.006111 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application with Lock( May 19 11:58:55.006111 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 11:58:55.006111 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._enter() May 19 11:58:55.006111 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 11:58:55.006111 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 11:58:55.006111 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 11:58:55.006111 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.creator() May 19 11:58:55.006111 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 11:58:55.006111 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application created_value = creator( May 19 11:58:55.007074 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 11:58:55.007074 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 11:58:55.007074 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 11:58:55.007074 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 11:58:55.007074 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 11:58:55.007074 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 11:58:55.007074 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 11:58:55.007074 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application May 19 11:58:55.011963 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 480/950] 10.222.0.23 () {56 vars in 1111 bytes} [Thu May 19 11:58:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 11:58:55.014649 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca296983-3ebd-467e-98dd-a538d10ebe40 None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:55.034352 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8950a038-4cc2-481c-9656-ffd5e45eafca None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:55.089265 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10510a84-2b2a-4e03-9877-0cb06c694e3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:55.090299 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10510a84-2b2a-4e03-9877-0cb06c694e3d None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:55.090583 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10510a84-2b2a-4e03-9877-0cb06c694e3d None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:55.090908 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10510a84-2b2a-4e03-9877-0cb06c694e3d None admin] PATH_INFO: `/v3/services` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:55.092851 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10510a84-2b2a-4e03-9877-0cb06c694e3d None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 11:58:55.093754 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10510a84-2b2a-4e03-9877-0cb06c694e3d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:55.095201 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10510a84-2b2a-4e03-9877-0cb06c694e3d None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:55.103085 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 471/951] 10.222.0.23 () {56 vars in 1122 bytes} [Thu May 19 11:58:54 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:58:55.112906 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10510a84-2b2a-4e03-9877-0cb06c694e3d None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:55.173398 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca296983-3ebd-467e-98dd-a538d10ebe40 None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:55.208113 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:58:55.214141 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14bf8e15-8f07-41e7-9903-6ffeced31f3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:55.214814 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bf8e15-8f07-41e7-9903-6ffeced31f3d None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:55.215101 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bf8e15-8f07-41e7-9903-6ffeced31f3d None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:55.215377 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bf8e15-8f07-41e7-9903-6ffeced31f3d None admin] PATH_INFO: `/v3/projects` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:55.218220 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14bf8e15-8f07-41e7-9903-6ffeced31f3d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 11:58:55.218220 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14bf8e15-8f07-41e7-9903-6ffeced31f3d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:55.219640 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14bf8e15-8f07-41e7-9903-6ffeced31f3d None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:55.233035 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 481/952] 10.222.0.23 () {56 vars in 1148 bytes} [Thu May 19 11:58:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 11:58:55.239188 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14bf8e15-8f07-41e7-9903-6ffeced31f3d None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:55.244911 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10510a84-2b2a-4e03-9877-0cb06c694e3d None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:55.287621 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b095ede-ba53-4774-b32a-d3be6db159f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:55.288642 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b095ede-ba53-4774-b32a-d3be6db159f4 None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:55.289046 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b095ede-ba53-4774-b32a-d3be6db159f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:55.289200 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b095ede-ba53-4774-b32a-d3be6db159f4 None admin] PATH_INFO: `/v3/services` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:55.289917 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b095ede-ba53-4774-b32a-d3be6db159f4 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 11:58:55.290274 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b095ede-ba53-4774-b32a-d3be6db159f4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:55.291655 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b095ede-ba53-4774-b32a-d3be6db159f4 None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:55.297279 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 472/953] 10.222.0.23 () {56 vars in 1122 bytes} [Thu May 19 11:58:55 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:58:55.298544 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597609c8-d10a-4382-86de-7dc8d1b7d35b None None] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:55.298834 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597609c8-d10a-4382-86de-7dc8d1b7d35b None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:55.299125 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597609c8-d10a-4382-86de-7dc8d1b7d35b None None] PATH_INFO: `/` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:55.300525 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 473/954] 10.222.0.23 () {54 vars in 926 bytes} [Thu May 19 11:58:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 11:58:55.305519 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597609c8-d10a-4382-86de-7dc8d1b7d35b None None] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:55.316981 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:58:55.354253 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14bf8e15-8f07-41e7-9903-6ffeced31f3d None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:55.394013 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8561d7ea-3837-40e3-92ab-54ce2cbd7eb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:55.395074 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8561d7ea-3837-40e3-92ab-54ce2cbd7eb6 None admin] REQUEST_METHOD: `PUT` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:55.395365 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8561d7ea-3837-40e3-92ab-54ce2cbd7eb6 None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:55.396054 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8561d7ea-3837-40e3-92ab-54ce2cbd7eb6 None admin] PATH_INFO: `/v3/projects/d70d25bf69314820b8aaecb65576978a/users/0c3181fb4f584a549a786adfefdf3dfb/roles/ff2dfc5f80304d9cbd3320d8ceffe2a2` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:55.398510 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8561d7ea-3837-40e3-92ab-54ce2cbd7eb6 None admin] RBAC: Authorizing `identity:create_grant(project_id=d70d25bf69314820b8aaecb65576978a, user_id=0c3181fb4f584a549a786adfefdf3dfb, role_id=ff2dfc5f80304d9cbd3320d8ceffe2a2)` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:55.400014 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8561d7ea-3837-40e3-92ab-54ce2cbd7eb6 None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:55.414813 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597609c8-d10a-4382-86de-7dc8d1b7d35b None None] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:55.451733 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12dbd30a-5c9c-4845-9964-afff6bbf691a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:55.452565 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12dbd30a-5c9c-4845-9964-afff6bbf691a None admin] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:55.452875 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12dbd30a-5c9c-4845-9964-afff6bbf691a None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:55.453190 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12dbd30a-5c9c-4845-9964-afff6bbf691a None admin] PATH_INFO: `/v3/endpoints` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:55.453934 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12dbd30a-5c9c-4845-9964-afff6bbf691a None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:55.455293 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12dbd30a-5c9c-4845-9964-afff6bbf691a None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:55.473474 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 482/955] 10.222.0.23 () {58 vars in 1439 bytes} [Thu May 19 11:58:55 2022] PUT /identity/v3/projects/d70d25bf69314820b8aaecb65576978a/users/0c3181fb4f584a549a786adfefdf3dfb/roles/ff2dfc5f80304d9cbd3320d8ceffe2a2 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 11:58:55.474681 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3ddbc1-f859-41a0-95b5-85e81bc068d7 None None] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:55.474971 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3ddbc1-f859-41a0-95b5-85e81bc068d7 None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:55.475250 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3ddbc1-f859-41a0-95b5-85e81bc068d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:55.494269 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.auth.core [None req-8e3ddbc1-f859-41a0-95b5-85e81bc068d7 None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 11:58:55.496040 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-8e3ddbc1-f859-41a0-95b5-85e81bc068d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 11:58:55.531821 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 474/956] 10.222.0.23 () {60 vars in 1144 bytes} [Thu May 19 11:58:55 2022] POST /identity/v3/endpoints => generated 349 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 11:58:55.574932 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:58:55.609998 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 483/957] 10.222.0.23 () {58 vars in 1023 bytes} [Thu May 19 11:58:55 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 11:58:55.633710 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12dbd30a-5c9c-4845-9964-afff6bbf691a None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:55.711940 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:58:55.756341 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12dbd30a-5c9c-4845-9964-afff6bbf691a None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:55.795210 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edd2779-9cec-4730-8b08-814e22626473 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:55.796540 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edd2779-9cec-4730-8b08-814e22626473 None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:55.796828 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edd2779-9cec-4730-8b08-814e22626473 None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:55.797110 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edd2779-9cec-4730-8b08-814e22626473 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:55.797821 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2edd2779-9cec-4730-8b08-814e22626473 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:55.799161 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2edd2779-9cec-4730-8b08-814e22626473 None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:55.811985 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application [None req-2edd2779-9cec-4730-8b08-814e22626473 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. May 19 11:58:55.811985 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 11:58:55.811985 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 11:58:55.811985 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 11:58:55.811985 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 11:58:55.811985 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 11:58:55.811985 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 11:58:55.811985 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 11:58:55.811985 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 19 11:58:55.811985 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application May 19 11:58:55.811985 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 11:58:55.811985 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application May 19 11:58:55.811985 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 11:58:55.811985 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 11:58:55.811985 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 11:58:55.811985 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 11:58:55.811985 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 11:58:55.813100 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 11:58:55.813100 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 11:58:55.813100 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 11:58:55.813100 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 11:58:55.813100 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 11:58:55.813100 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 11:58:55.813100 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 11:58:55.813100 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 11:58:55.813100 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 11:58:55.813100 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 11:58:55.813100 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 11:58:55.813100 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 11:58:55.813100 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 11:58:55.813100 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 11:58:55.813100 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 11:58:55.813100 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.get_or_create( May 19 11:58:55.813940 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 11:58:55.813940 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application with Lock( May 19 11:58:55.813940 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 11:58:55.813940 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._enter() May 19 11:58:55.813940 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 11:58:55.813940 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 11:58:55.813940 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 11:58:55.813940 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.creator() May 19 11:58:55.813940 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 11:58:55.813940 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application created_value = creator( May 19 11:58:55.813940 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 11:58:55.813940 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 11:58:55.813940 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 19 11:58:55.813940 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application May 19 11:58:55.817292 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 475/958] 10.222.0.23 () {56 vars in 1111 bytes} [Thu May 19 11:58:55 2022] GET /identity/v3/services/compute => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 11:58:55.824899 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3ddbc1-f859-41a0-95b5-85e81bc068d7 None None] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:55.933512 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3ddbc1-f859-41a0-95b5-85e81bc068d7 None None] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:55.971294 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59909b1-0a8c-48c5-86b9-71ac3b73c65b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:55.972061 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59909b1-0a8c-48c5-86b9-71ac3b73c65b None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:55.972346 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59909b1-0a8c-48c5-86b9-71ac3b73c65b None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:55.972625 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59909b1-0a8c-48c5-86b9-71ac3b73c65b None admin] PATH_INFO: `/v3/services` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:55.973334 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59909b1-0a8c-48c5-86b9-71ac3b73c65b None admin] RBAC: Adding query filter params (name=compute) {{(pid=80540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 11:58:55.973640 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59909b1-0a8c-48c5-86b9-71ac3b73c65b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:55.975027 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59909b1-0a8c-48c5-86b9-71ac3b73c65b None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:55.980482 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 484/959] 10.222.0.23 () {56 vars in 1112 bytes} [Thu May 19 11:58:55 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:58:55.986798 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edd2779-9cec-4730-8b08-814e22626473 None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:56.097011 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edd2779-9cec-4730-8b08-814e22626473 None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:56.127827 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:58:56.136433 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edcfda65-09df-4437-81f6-fe0e184694d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:56.144603 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcfda65-09df-4437-81f6-fe0e184694d8 None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:56.144603 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcfda65-09df-4437-81f6-fe0e184694d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:56.144603 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcfda65-09df-4437-81f6-fe0e184694d8 None admin] PATH_INFO: `/v3/services` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:56.144603 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edcfda65-09df-4437-81f6-fe0e184694d8 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 11:58:56.144603 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edcfda65-09df-4437-81f6-fe0e184694d8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:56.144603 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edcfda65-09df-4437-81f6-fe0e184694d8 None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:56.145499 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 476/960] 10.222.0.23 () {56 vars in 1112 bytes} [Thu May 19 11:58:55 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:58:57.358140 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17229606-a7e0-42eb-84fe-a4b4f020ec84 None None] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:57.358656 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17229606-a7e0-42eb-84fe-a4b4f020ec84 None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:57.358734 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17229606-a7e0-42eb-84fe-a4b4f020ec84 None None] PATH_INFO: `/` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:57.360372 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 485/961] 10.222.0.23 () {54 vars in 926 bytes} [Thu May 19 11:58:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 11:58:57.365927 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75651389-677d-4ae9-85dd-c7ea9a0a8b24 None None] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:57.366699 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75651389-677d-4ae9-85dd-c7ea9a0a8b24 None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:57.367130 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75651389-677d-4ae9-85dd-c7ea9a0a8b24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:57.385618 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.auth.core [None req-75651389-677d-4ae9-85dd-c7ea9a0a8b24 None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 11:58:57.387085 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-75651389-677d-4ae9-85dd-c7ea9a0a8b24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 11:58:57.461002 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 477/962] 10.222.0.23 () {58 vars in 1023 bytes} [Thu May 19 11:58:57 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 11:58:57.486512 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17229606-a7e0-42eb-84fe-a4b4f020ec84 None None] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:57.534325 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:58:57.595493 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17229606-a7e0-42eb-84fe-a4b4f020ec84 None None] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:57.635954 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f613bd-af66-4242-97e3-ede752fee4fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:57.636963 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f613bd-af66-4242-97e3-ede752fee4fd None admin] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:57.637449 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f613bd-af66-4242-97e3-ede752fee4fd None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:57.637926 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f613bd-af66-4242-97e3-ede752fee4fd None admin] PATH_INFO: `/v3/users` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:57.638910 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f613bd-af66-4242-97e3-ede752fee4fd None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:57.640782 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f613bd-af66-4242-97e3-ede752fee4fd None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:57.690223 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.sql.core [None req-79f613bd-af66-4242-97e3-ede752fee4fd None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") May 19 11:58:57.690223 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] May 19 11:58:57.690223 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [parameters: {'user_id': '494b6a750480462d9bd3b584bbbd2dd1', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] May 19 11:58:57.690223 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80540) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} May 19 11:58:57.698616 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application [None req-79f613bd-af66-4242-97e3-ede752fee4fd None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. May 19 11:58:57.698616 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 11:58:57.698616 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 19 11:58:57.698616 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application self.dialect.do_execute( May 19 11:58:57.698616 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 19 11:58:57.698616 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 19 11:58:57.698616 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 19 11:58:57.698616 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application result = self._query(query) May 19 11:58:57.698616 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 19 11:58:57.698616 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application conn.query(q) May 19 11:58:57.698616 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 19 11:58:57.698616 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 19 11:58:57.698616 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 19 11:58:57.698616 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application result.read() May 19 11:58:57.698616 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 19 11:58:57.699501 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 19 11:58:57.699501 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 19 11:58:57.699501 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application packet.raise_for_error() May 19 11:58:57.699501 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 19 11:58:57.699501 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 19 11:58:57.699501 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 19 11:58:57.699501 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 19 11:58:57.699501 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") May 19 11:58:57.699501 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application May 19 11:58:57.699501 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: May 19 11:58:57.699501 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application May 19 11:58:57.699501 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 11:58:57.699501 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper May 19 11:58:57.699501 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return method(*args, **kwargs) May 19 11:58:57.699501 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user May 19 11:58:57.699501 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) May 19 11:58:57.699501 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 19 11:58:57.699501 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application next(self.gen) May 19 11:58:57.700618 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope May 19 11:58:57.700618 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application yield resource May 19 11:58:57.700618 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 19 11:58:57.700618 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application next(self.gen) May 19 11:58:57.700618 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session May 19 11:58:57.700618 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application self.session.rollback() May 19 11:58:57.700618 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ May 19 11:58:57.700618 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application self.force_reraise() May 19 11:58:57.700618 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise May 19 11:58:57.700618 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application raise self.value May 19 11:58:57.700618 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session May 19 11:58:57.700618 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) May 19 11:58:57.700618 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction May 19 11:58:57.700618 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application session.commit() May 19 11:58:57.700618 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1435, in commit May 19 11:58:57.700618 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) May 19 11:58:57.700618 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit May 19 11:58:57.701529 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application self._prepare_impl() May 19 11:58:57.701529 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl May 19 11:58:57.701529 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application self.session.flush() May 19 11:58:57.701529 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3367, in flush May 19 11:58:57.701529 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application self._flush(objects) May 19 11:58:57.701529 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3507, in _flush May 19 11:58:57.701529 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) May 19 11:58:57.701529 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ May 19 11:58:57.701529 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application compat.raise_( May 19 11:58:57.701529 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 19 11:58:57.701529 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application raise exception May 19 11:58:57.701529 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3467, in _flush May 19 11:58:57.701529 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application flush_context.execute() May 19 11:58:57.701529 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute May 19 11:58:57.701529 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application rec.execute(self) May 19 11:58:57.701529 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute May 19 11:58:57.701529 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( May 19 11:58:57.702431 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj May 19 11:58:57.702431 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application _emit_insert_statements( May 19 11:58:57.702431 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements May 19 11:58:57.702431 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application result = connection._execute_20( May 19 11:58:57.702431 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 May 19 11:58:57.702431 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) May 19 11:58:57.702431 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection May 19 11:58:57.702431 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return connection._execute_clauseelement( May 19 11:58:57.702431 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement May 19 11:58:57.702431 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application ret = self._execute_context( May 19 11:58:57.702431 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context May 19 11:58:57.702431 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application self._handle_dbapi_exception( May 19 11:58:57.702431 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception May 19 11:58:57.702431 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) May 19 11:58:57.702431 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 19 11:58:57.702431 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application raise exception May 19 11:58:57.703423 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 19 11:58:57.703423 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application self.dialect.do_execute( May 19 11:58:57.703423 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 19 11:58:57.703423 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 19 11:58:57.703423 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 19 11:58:57.703423 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application result = self._query(query) May 19 11:58:57.703423 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 19 11:58:57.703423 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application conn.query(q) May 19 11:58:57.703423 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 19 11:58:57.703423 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 19 11:58:57.703423 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 19 11:58:57.703423 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application result.read() May 19 11:58:57.703423 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 19 11:58:57.703423 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 19 11:58:57.703423 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 19 11:58:57.703423 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application packet.raise_for_error() May 19 11:58:57.706203 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 19 11:58:57.706203 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 19 11:58:57.706203 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 19 11:58:57.706203 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 19 11:58:57.706203 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") May 19 11:58:57.706203 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] May 19 11:58:57.706203 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application [parameters: {'user_id': '494b6a750480462d9bd3b584bbbd2dd1', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] May 19 11:58:57.706203 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) May 19 11:58:57.706203 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application May 19 11:58:57.706203 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 11:58:57.706203 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application May 19 11:58:57.706203 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 11:58:57.706203 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 11:58:57.706203 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 11:58:57.706203 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 11:58:57.708174 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 11:58:57.708174 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 11:58:57.708174 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 11:58:57.708174 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 11:58:57.708174 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 11:58:57.708174 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 11:58:57.708174 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 11:58:57.708174 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post May 19 11:58:57.708174 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( May 19 11:58:57.708174 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 11:58:57.708174 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 11:58:57.708174 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 11:58:57.708174 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 11:58:57.708174 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 11:58:57.708174 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 11:58:57.708174 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user May 19 11:58:57.708174 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) May 19 11:58:57.710313 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects May 19 11:58:57.710313 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) May 19 11:58:57.710313 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper May 19 11:58:57.710313 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, May 19 11:58:57.710313 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. May 19 11:58:57.710313 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application May 19 11:58:57.713090 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 486/963] 10.222.0.23 () {60 vars in 1131 bytes} [Thu May 19 11:58:57 2022] POST /identity/v3/users => generated 160 bytes in 227 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) May 19 11:58:57.719408 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75651389-677d-4ae9-85dd-c7ea9a0a8b24 None None] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:57.788754 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7c35ac-c1eb-49ac-aeb8-3e0346cee7a5 None None] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:57.789220 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7c35ac-c1eb-49ac-aeb8-3e0346cee7a5 None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:57.789571 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7c35ac-c1eb-49ac-aeb8-3e0346cee7a5 None None] PATH_INFO: `/` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:57.797207 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 487/964] 10.222.0.23 () {54 vars in 926 bytes} [Thu May 19 11:58:57 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 11:58:57.803355 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cf4012-7107-4dda-b1f9-7498611f3f7a None None] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:57.803696 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cf4012-7107-4dda-b1f9-7498611f3f7a None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:57.803955 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cf4012-7107-4dda-b1f9-7498611f3f7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:57.823634 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.auth.core [None req-26cf4012-7107-4dda-b1f9-7498611f3f7a None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 11:58:57.825300 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-26cf4012-7107-4dda-b1f9-7498611f3f7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 11:58:57.840244 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75651389-677d-4ae9-85dd-c7ea9a0a8b24 None None] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:57.870820 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:58:57.885546 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937935b6-6780-44bf-a6aa-67f771081e49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:57.886569 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937935b6-6780-44bf-a6aa-67f771081e49 None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:57.886798 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937935b6-6780-44bf-a6aa-67f771081e49 None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:57.887078 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937935b6-6780-44bf-a6aa-67f771081e49 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:57.898682 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937935b6-6780-44bf-a6aa-67f771081e49 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:57.900583 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937935b6-6780-44bf-a6aa-67f771081e49 None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:57.907816 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 488/965] 10.222.0.23 () {58 vars in 1023 bytes} [Thu May 19 11:58:57 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 11:58:57.918375 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application [None req-937935b6-6780-44bf-a6aa-67f771081e49 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 19 11:58:57.918375 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 11:58:57.918375 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 11:58:57.918375 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 11:58:57.918375 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 11:58:57.918375 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 11:58:57.918375 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 11:58:57.918375 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 11:58:57.918375 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 11:58:57.918375 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 11:58:57.918375 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 11:58:57.918375 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 11:58:57.918375 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 11:58:57.918375 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 11:58:57.918375 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 11:58:57.918375 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 11:58:57.919301 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 11:58:57.919301 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 11:58:57.919301 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 11:58:57.919301 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 11:58:57.919301 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 11:58:57.919301 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 11:58:57.919301 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 11:58:57.919301 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 11:58:57.919301 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 11:58:57.919301 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.get_or_create( May 19 11:58:57.919301 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 11:58:57.919301 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application with Lock( May 19 11:58:57.919301 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 11:58:57.919301 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._enter() May 19 11:58:57.919301 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 11:58:57.919301 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 11:58:57.919301 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 11:58:57.919301 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.creator() May 19 11:58:57.921230 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 11:58:57.921230 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application created_value = creator( May 19 11:58:57.921230 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 11:58:57.921230 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 11:58:57.921230 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 11:58:57.921230 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 11:58:57.921230 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 11:58:57.921230 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 11:58:57.921230 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 19 11:58:57.921230 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application May 19 11:58:57.921230 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 478/966] 10.222.0.23 () {56 vars in 1099 bytes} [Thu May 19 11:58:57 2022] GET /identity/v3/users/cinder => generated 84 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 11:58:57.926273 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26cf4012-7107-4dda-b1f9-7498611f3f7a None None] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:57.932862 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937935b6-6780-44bf-a6aa-67f771081e49 None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:57.984166 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:58:58.048518 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26cf4012-7107-4dda-b1f9-7498611f3f7a None None] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:58.051034 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937935b6-6780-44bf-a6aa-67f771081e49 None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:58.086565 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556afeeb-f81f-4ec2-951d-2b42db23d5af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:58.087248 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556afeeb-f81f-4ec2-951d-2b42db23d5af None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:58.087541 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556afeeb-f81f-4ec2-951d-2b42db23d5af None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:58.087663 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ad90b0-e845-444d-8258-36dcef7b9104 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:58.087880 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556afeeb-f81f-4ec2-951d-2b42db23d5af None admin] PATH_INFO: `/v3/users` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:58.088444 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ad90b0-e845-444d-8258-36dcef7b9104 None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:58.088661 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556afeeb-f81f-4ec2-951d-2b42db23d5af None admin] RBAC: Adding query filter params (name=cinder) {{(pid=80540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 11:58:58.088757 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ad90b0-e845-444d-8258-36dcef7b9104 None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:58.088973 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556afeeb-f81f-4ec2-951d-2b42db23d5af None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:58.089089 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ad90b0-e845-444d-8258-36dcef7b9104 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:58.090601 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556afeeb-f81f-4ec2-951d-2b42db23d5af None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:58.100759 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ad90b0-e845-444d-8258-36dcef7b9104 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 11:58:58.109473 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application [None req-92ad90b0-e845-444d-8258-36dcef7b9104 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 19 11:58:58.109473 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 11:58:58.109473 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 11:58:58.109473 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 11:58:58.109473 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 11:58:58.109473 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 11:58:58.109473 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 11:58:58.109473 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 11:58:58.109473 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 11:58:58.109473 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 11:58:58.109473 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 11:58:58.109473 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 11:58:58.109473 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 11:58:58.109473 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 11:58:58.109473 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 11:58:58.109473 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 11:58:58.110446 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 11:58:58.110446 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 11:58:58.110446 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 11:58:58.110446 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 11:58:58.110446 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 11:58:58.110446 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 11:58:58.110446 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 11:58:58.110446 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 11:58:58.110446 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 11:58:58.110446 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.get_or_create( May 19 11:58:58.110446 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 11:58:58.110446 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application with Lock( May 19 11:58:58.110446 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 11:58:58.110446 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._enter() May 19 11:58:58.110446 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 11:58:58.110446 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 11:58:58.116870 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 11:58:58.116870 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.creator() May 19 11:58:58.116870 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 11:58:58.116870 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application created_value = creator( May 19 11:58:58.116870 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 11:58:58.116870 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 11:58:58.116870 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 11:58:58.116870 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 11:58:58.116870 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 11:58:58.116870 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 11:58:58.116870 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 19 11:58:58.116870 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application May 19 11:58:58.116870 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 479/967] 10.222.0.23 () {56 vars in 1120 bytes} [Thu May 19 11:58:57 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 11:58:58.116870 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665f02d6-845c-4b1d-8285-c1e48a5aacfe None None] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:58.118018 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665f02d6-845c-4b1d-8285-c1e48a5aacfe None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:58.118018 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665f02d6-845c-4b1d-8285-c1e48a5aacfe None None] PATH_INFO: `/` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:58.119212 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 480/968] 10.222.0.23 () {54 vars in 926 bytes} [Thu May 19 11:58:58 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 11:58:58.119212 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665f02d6-845c-4b1d-8285-c1e48a5aacfe None None] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:58.133122 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 489/969] 10.222.0.23 () {56 vars in 1101 bytes} [Thu May 19 11:58:57 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:58:58.135493 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a330db-abd9-4727-9246-2ab9b34d7e1b None None] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:58.135996 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a330db-abd9-4727-9246-2ab9b34d7e1b None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:58.136755 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a330db-abd9-4727-9246-2ab9b34d7e1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:58.161043 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.auth.core [None req-98a330db-abd9-4727-9246-2ab9b34d7e1b None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 11:58:58.163202 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-98a330db-abd9-4727-9246-2ab9b34d7e1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 11:58:58.178024 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:58:58.255910 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 490/970] 10.222.0.23 () {58 vars in 1023 bytes} [Thu May 19 11:58:58 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 11:58:58.257780 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665f02d6-845c-4b1d-8285-c1e48a5aacfe None None] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:58.272904 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a330db-abd9-4727-9246-2ab9b34d7e1b None None] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:58.307987 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:58:58.308716 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b78916f-08d7-4877-a0fe-a6309db5c461 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:58.311466 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b78916f-08d7-4877-a0fe-a6309db5c461 None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:58.312020 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b78916f-08d7-4877-a0fe-a6309db5c461 None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:58.312642 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b78916f-08d7-4877-a0fe-a6309db5c461 None admin] PATH_INFO: `/v3/roles` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:58.314152 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b78916f-08d7-4877-a0fe-a6309db5c461 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 11:58:58.314914 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b78916f-08d7-4877-a0fe-a6309db5c461 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:58.316840 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b78916f-08d7-4877-a0fe-a6309db5c461 None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:58.335908 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 481/971] 10.222.0.23 () {56 vars in 1115 bytes} [Thu May 19 11:58:58 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:58:58.337662 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b78916f-08d7-4877-a0fe-a6309db5c461 None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:58.415930 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a330db-abd9-4727-9246-2ab9b34d7e1b None None] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:58.458867 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8a0cef-7a38-4019-9ce9-113b4e230f04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:58.459994 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8a0cef-7a38-4019-9ce9-113b4e230f04 None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:58.460290 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8a0cef-7a38-4019-9ce9-113b4e230f04 None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:58.460579 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8a0cef-7a38-4019-9ce9-113b4e230f04 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:58.461683 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8a0cef-7a38-4019-9ce9-113b4e230f04 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:58.463283 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8a0cef-7a38-4019-9ce9-113b4e230f04 None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:58.464189 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b78916f-08d7-4877-a0fe-a6309db5c461 None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:58.468846 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:58:58.477678 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application [None req-5f8a0cef-7a38-4019-9ce9-113b4e230f04 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 11:58:58.477678 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 11:58:58.477678 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 11:58:58.477678 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 11:58:58.477678 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 11:58:58.477678 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 11:58:58.477678 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 11:58:58.477678 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 11:58:58.477678 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 11:58:58.477678 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application May 19 11:58:58.477678 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 11:58:58.477678 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application May 19 11:58:58.477678 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 11:58:58.477678 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 11:58:58.477678 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 11:58:58.477678 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 11:58:58.477678 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 11:58:58.482640 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 11:58:58.482640 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 11:58:58.482640 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 11:58:58.482640 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 11:58:58.482640 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 11:58:58.482640 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 11:58:58.482640 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 11:58:58.482640 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 11:58:58.482640 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 11:58:58.482640 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 11:58:58.482640 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 11:58:58.482640 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 11:58:58.482640 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 11:58:58.482640 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 11:58:58.482640 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 11:58:58.482640 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.get_or_create( May 19 11:58:58.483898 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 11:58:58.483898 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application with Lock( May 19 11:58:58.483898 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 11:58:58.483898 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._enter() May 19 11:58:58.483898 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 11:58:58.483898 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 11:58:58.483898 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 11:58:58.483898 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.creator() May 19 11:58:58.483898 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 11:58:58.483898 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application created_value = creator( May 19 11:58:58.483898 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 11:58:58.483898 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 11:58:58.483898 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 11:58:58.483898 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application May 19 11:58:58.483898 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 491/972] 10.222.0.23 () {56 vars in 1126 bytes} [Thu May 19 11:58:58 2022] GET /identity/v3/services/object-store => generated 93 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 11:58:58.486643 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8a0cef-7a38-4019-9ce9-113b4e230f04 None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:58.506167 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c02069-5106-4023-86f5-b6151fc8373b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:58.506961 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c02069-5106-4023-86f5-b6151fc8373b None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:58.507241 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c02069-5106-4023-86f5-b6151fc8373b None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:58.507514 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c02069-5106-4023-86f5-b6151fc8373b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:58.517900 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c02069-5106-4023-86f5-b6151fc8373b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:58.519340 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c02069-5106-4023-86f5-b6151fc8373b None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:58.538701 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application [None req-60c02069-5106-4023-86f5-b6151fc8373b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 11:58:58.538701 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 11:58:58.538701 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 11:58:58.538701 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 11:58:58.538701 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 11:58:58.538701 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 11:58:58.538701 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 11:58:58.538701 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 11:58:58.538701 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 11:58:58.538701 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 11:58:58.538701 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 11:58:58.538701 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 11:58:58.538701 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 11:58:58.538701 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 11:58:58.538701 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 11:58:58.539825 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 11:58:58.539825 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 11:58:58.539825 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 11:58:58.539825 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 11:58:58.539825 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 11:58:58.539825 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 11:58:58.539825 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.get_or_create( May 19 11:58:58.539825 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 11:58:58.539825 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application with Lock( May 19 11:58:58.539825 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 11:58:58.539825 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._enter() May 19 11:58:58.539825 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 11:58:58.539825 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 11:58:58.539825 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 11:58:58.539825 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.creator() May 19 11:58:58.539825 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 11:58:58.539825 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application created_value = creator( May 19 11:58:58.540930 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 11:58:58.540930 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 11:58:58.540930 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 11:58:58.540930 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application May 19 11:58:58.540930 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 482/973] 10.222.0.23 () {56 vars in 1108 bytes} [Thu May 19 11:58:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 11:58:58.541270 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53369230-fe14-49de-a181-c19255026ba8 None None] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:58.541409 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53369230-fe14-49de-a181-c19255026ba8 None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:58.541685 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53369230-fe14-49de-a181-c19255026ba8 None None] PATH_INFO: `/` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:58.542932 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 483/974] 10.222.0.23 () {54 vars in 926 bytes} [Thu May 19 11:58:58 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 11:58:58.548545 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53369230-fe14-49de-a181-c19255026ba8 None None] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:58.580683 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:58:58.595925 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8a0cef-7a38-4019-9ce9-113b4e230f04 None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:58.630090 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8092a0-a9d4-4a3f-8728-bec0c94dd031 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:58.630758 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8092a0-a9d4-4a3f-8728-bec0c94dd031 None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:58.631039 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8092a0-a9d4-4a3f-8728-bec0c94dd031 None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:58.631314 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8092a0-a9d4-4a3f-8728-bec0c94dd031 None admin] PATH_INFO: `/v3/services` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:58.632058 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8092a0-a9d4-4a3f-8728-bec0c94dd031 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 11:58:58.632381 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8092a0-a9d4-4a3f-8728-bec0c94dd031 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:58.633753 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8092a0-a9d4-4a3f-8728-bec0c94dd031 None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:58.638754 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 492/975] 10.222.0.23 () {56 vars in 1122 bytes} [Thu May 19 11:58:58 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:58:58.639752 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efebf6f-d6b5-4799-920f-8ea7834fbb10 None None] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:58.640183 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efebf6f-d6b5-4799-920f-8ea7834fbb10 None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:58.640349 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efebf6f-d6b5-4799-920f-8ea7834fbb10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:58.656959 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.auth.core [None req-2efebf6f-d6b5-4799-920f-8ea7834fbb10 None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 11:58:58.657855 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53369230-fe14-49de-a181-c19255026ba8 None None] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:58.658165 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-2efebf6f-d6b5-4799-920f-8ea7834fbb10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 11:58:58.691948 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e693e34e-bab8-4937-9531-6bf34c0b60f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:58.692530 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e693e34e-bab8-4937-9531-6bf34c0b60f4 None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:58.692816 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e693e34e-bab8-4937-9531-6bf34c0b60f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:58.693096 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e693e34e-bab8-4937-9531-6bf34c0b60f4 None admin] PATH_INFO: `/v3/domains` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:58.693821 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e693e34e-bab8-4937-9531-6bf34c0b60f4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 11:58:58.694128 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e693e34e-bab8-4937-9531-6bf34c0b60f4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:58.695492 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e693e34e-bab8-4937-9531-6bf34c0b60f4 None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:58.707506 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 484/976] 10.222.0.23 () {56 vars in 1109 bytes} [Thu May 19 11:58:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:58:58.708355 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e693e34e-bab8-4937-9531-6bf34c0b60f4 None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:58.726600 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 493/977] 10.222.0.23 () {58 vars in 1023 bytes} [Thu May 19 11:58:58 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 11:58:58.727456 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efebf6f-d6b5-4799-920f-8ea7834fbb10 None None] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:58.744003 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:58:58.812083 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:58:58.828555 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e693e34e-bab8-4937-9531-6bf34c0b60f4 None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:58.832590 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efebf6f-d6b5-4799-920f-8ea7834fbb10 None None] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:58.862266 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d69331-9194-4e6a-a4cb-83a04305029d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:58.862935 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d69331-9194-4e6a-a4cb-83a04305029d None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:58.863222 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d69331-9194-4e6a-a4cb-83a04305029d None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:58.863505 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d69331-9194-4e6a-a4cb-83a04305029d None admin] PATH_INFO: `/v3/services` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:58.864263 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d69331-9194-4e6a-a4cb-83a04305029d None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 11:58:58.864586 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d69331-9194-4e6a-a4cb-83a04305029d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:58.865962 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d69331-9194-4e6a-a4cb-83a04305029d None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:58.867604 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37529a3-f33f-4742-814e-ef2e52c25a03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:58.868452 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37529a3-f33f-4742-814e-ef2e52c25a03 None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:58.868739 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37529a3-f33f-4742-814e-ef2e52c25a03 None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:58.869021 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37529a3-f33f-4742-814e-ef2e52c25a03 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:58.870956 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 485/978] 10.222.0.23 () {56 vars in 1122 bytes} [Thu May 19 11:58:58 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:58:58.872107 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d69331-9194-4e6a-a4cb-83a04305029d None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:58.876938 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37529a3-f33f-4742-814e-ef2e52c25a03 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:58.878425 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37529a3-f33f-4742-814e-ef2e52c25a03 None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:58.894241 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application [None req-d37529a3-f33f-4742-814e-ef2e52c25a03 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 19 11:58:58.894241 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 11:58:58.894241 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 11:58:58.894241 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 11:58:58.894241 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 11:58:58.894241 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 11:58:58.894241 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 11:58:58.894241 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 11:58:58.894241 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 11:58:58.894241 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 11:58:58.894241 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 11:58:58.894241 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 11:58:58.894241 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 11:58:58.894241 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 11:58:58.894241 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 11:58:58.894241 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 11:58:58.895435 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 11:58:58.895435 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 11:58:58.895435 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 11:58:58.895435 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 11:58:58.895435 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 11:58:58.895435 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 11:58:58.895435 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 11:58:58.895435 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 11:58:58.895435 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 11:58:58.895435 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.get_or_create( May 19 11:58:58.895435 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 11:58:58.895435 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application with Lock( May 19 11:58:58.895435 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 11:58:58.895435 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._enter() May 19 11:58:58.895435 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 11:58:58.895435 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 11:58:58.895435 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 11:58:58.895435 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.creator() May 19 11:58:58.896447 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 11:58:58.896447 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application created_value = creator( May 19 11:58:58.896447 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 11:58:58.896447 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 11:58:58.896447 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 11:58:58.896447 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 11:58:58.896447 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 11:58:58.896447 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 11:58:58.896447 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 19 11:58:58.896447 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application May 19 11:58:58.896447 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 494/979] 10.222.0.23 () {56 vars in 1117 bytes} [Thu May 19 11:58:58 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 11:58:58.896447 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37529a3-f33f-4742-814e-ef2e52c25a03 None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:58.963877 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:58:58.982612 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d69331-9194-4e6a-a4cb-83a04305029d None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:58.996456 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37529a3-f33f-4742-814e-ef2e52c25a03 None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:59.017816 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbe01f1-5934-426d-9dad-12f9349cf340 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:59.018400 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbe01f1-5934-426d-9dad-12f9349cf340 None admin] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:59.018685 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbe01f1-5934-426d-9dad-12f9349cf340 None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:59.018960 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbe01f1-5934-426d-9dad-12f9349cf340 None admin] PATH_INFO: `/v3/services` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:59.019718 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bbe01f1-5934-426d-9dad-12f9349cf340 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:59.021136 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bbe01f1-5934-426d-9dad-12f9349cf340 None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:59.032002 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0da3b47-b23e-4a88-8db1-adcae305dcc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:59.032660 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0da3b47-b23e-4a88-8db1-adcae305dcc3 None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:59.032951 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0da3b47-b23e-4a88-8db1-adcae305dcc3 None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:59.033229 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0da3b47-b23e-4a88-8db1-adcae305dcc3 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:59.034047 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0da3b47-b23e-4a88-8db1-adcae305dcc3 None admin] RBAC: Adding query filter params (interface=admin, service_id=0ba01aa9ffd04ed18443612845ad5952, region_id=RegionOne) {{(pid=80540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 11:58:59.034346 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0da3b47-b23e-4a88-8db1-adcae305dcc3 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:59.035720 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0da3b47-b23e-4a88-8db1-adcae305dcc3 None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:59.040703 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 495/980] 10.222.0.23 () {56 vars in 1249 bytes} [Thu May 19 11:58:58 2022] GET /identity/v3/endpoints?service_id=0ba01aa9ffd04ed18443612845ad5952&interface=admin®ion_id=RegionOne => generated 193 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:58:59.044767 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0da3b47-b23e-4a88-8db1-adcae305dcc3 None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:59.045595 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:58:59.084686 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 486/981] 10.222.0.23 () {60 vars in 1141 bytes} [Thu May 19 11:58:58 2022] POST /identity/v3/services => generated 242 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 11:58:59.085521 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbe01f1-5934-426d-9dad-12f9349cf340 None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:59.154285 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0da3b47-b23e-4a88-8db1-adcae305dcc3 None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:59.204277 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8749fa53-4b24-4ffa-93cd-f39841226bb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:59.205066 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8749fa53-4b24-4ffa-93cd-f39841226bb0 None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:59.205361 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8749fa53-4b24-4ffa-93cd-f39841226bb0 None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:59.205643 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8749fa53-4b24-4ffa-93cd-f39841226bb0 None admin] PATH_INFO: `/v3/users` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:59.206506 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8749fa53-4b24-4ffa-93cd-f39841226bb0 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 11:58:59.206626 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbe01f1-5934-426d-9dad-12f9349cf340 None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:59.206837 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8749fa53-4b24-4ffa-93cd-f39841226bb0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:59.208337 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8749fa53-4b24-4ffa-93cd-f39841226bb0 None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:59.233832 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 496/982] 10.222.0.23 () {56 vars in 1149 bytes} [Thu May 19 11:58:59 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:58:59.235012 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8749fa53-4b24-4ffa-93cd-f39841226bb0 None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:59.254102 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122abe39-13a7-4078-83a7-ac93e4e78d44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:59.254794 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122abe39-13a7-4078-83a7-ac93e4e78d44 None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:59.255301 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122abe39-13a7-4078-83a7-ac93e4e78d44 None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:59.255667 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122abe39-13a7-4078-83a7-ac93e4e78d44 None admin] PATH_INFO: `/v3/services` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:59.256535 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122abe39-13a7-4078-83a7-ac93e4e78d44 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:59.257934 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122abe39-13a7-4078-83a7-ac93e4e78d44 None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:59.265127 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 487/983] 10.222.0.23 () {56 vars in 1087 bytes} [Thu May 19 11:58:59 2022] GET /identity/v3/services => generated 1727 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 11:58:59.313674 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:58:59.368528 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8749fa53-4b24-4ffa-93cd-f39841226bb0 None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:59.407303 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb783c8-719d-4e42-9b65-8ef49541f9b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:59.408012 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb783c8-719d-4e42-9b65-8ef49541f9b3 None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:59.408299 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb783c8-719d-4e42-9b65-8ef49541f9b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:59.408578 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb783c8-719d-4e42-9b65-8ef49541f9b3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:59.485939 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb783c8-719d-4e42-9b65-8ef49541f9b3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:59.487355 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb783c8-719d-4e42-9b65-8ef49541f9b3 None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:59.548212 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:58:59.593364 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 497/984] 10.222.0.23 () {58 vars in 1282 bytes} [Thu May 19 11:58:59 2022] GET /identity/v3/auth/tokens => generated 1560 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 19 11:58:59.598952 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122abe39-13a7-4078-83a7-ac93e4e78d44 None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:59.602815 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:58:59.706131 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122abe39-13a7-4078-83a7-ac93e4e78d44 None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:59.741572 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4780eb0-20f3-4936-9a69-d79f66371506 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:59.742293 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4780eb0-20f3-4936-9a69-d79f66371506 None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:59.742578 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4780eb0-20f3-4936-9a69-d79f66371506 None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:59.742855 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4780eb0-20f3-4936-9a69-d79f66371506 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:59.753288 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4780eb0-20f3-4936-9a69-d79f66371506 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:59.754750 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4780eb0-20f3-4936-9a69-d79f66371506 None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:59.773304 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application [None req-f4780eb0-20f3-4936-9a69-d79f66371506 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 11:58:59.773304 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 11:58:59.773304 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 11:58:59.773304 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 11:58:59.773304 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 11:58:59.773304 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 11:58:59.773304 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 11:58:59.773304 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 11:58:59.773304 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 11:58:59.773304 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 11:58:59.773304 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 11:58:59.773304 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 11:58:59.773304 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 11:58:59.773304 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 11:58:59.773304 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 11:58:59.774252 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 11:58:59.774252 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 11:58:59.774252 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 11:58:59.774252 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 11:58:59.774252 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 11:58:59.774252 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 11:58:59.774252 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.get_or_create( May 19 11:58:59.774252 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 11:58:59.774252 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application with Lock( May 19 11:58:59.774252 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 11:58:59.774252 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._enter() May 19 11:58:59.774252 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 11:58:59.774252 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 11:58:59.774252 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 11:58:59.774252 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.creator() May 19 11:58:59.774252 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 11:58:59.774252 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application created_value = creator( May 19 11:58:59.775204 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 11:58:59.775204 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 11:58:59.775204 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 11:58:59.775204 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application May 19 11:58:59.775204 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 488/985] 10.222.0.23 () {56 vars in 1108 bytes} [Thu May 19 11:58:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 11:58:59.781047 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb783c8-719d-4e42-9b65-8ef49541f9b3 None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:58:59.885730 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb783c8-719d-4e42-9b65-8ef49541f9b3 None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:58:59.922901 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0631508f-dc1f-4adf-87aa-4954744258ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:58:59.923570 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0631508f-dc1f-4adf-87aa-4954744258ff None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:58:59.923890 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0631508f-dc1f-4adf-87aa-4954744258ff None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:58:59.924177 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0631508f-dc1f-4adf-87aa-4954744258ff None admin] PATH_INFO: `/v3/domains` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:58:59.924905 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0631508f-dc1f-4adf-87aa-4954744258ff None admin] RBAC: Adding query filter params (name=Default) {{(pid=80540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 11:58:59.925222 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0631508f-dc1f-4adf-87aa-4954744258ff None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:58:59.927897 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0631508f-dc1f-4adf-87aa-4954744258ff None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:58:59.943385 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 498/986] 10.222.0.23 () {56 vars in 1109 bytes} [Thu May 19 11:58:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:58:59.950472 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4780eb0-20f3-4936-9a69-d79f66371506 None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:59:00.010856 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:59:00.056934 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4780eb0-20f3-4936-9a69-d79f66371506 None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:59:00.097120 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547f7209-bf04-4dad-8319-0bc21b287d7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:59:00.097989 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547f7209-bf04-4dad-8319-0bc21b287d7a None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:59:00.098272 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547f7209-bf04-4dad-8319-0bc21b287d7a None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:59:00.098550 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547f7209-bf04-4dad-8319-0bc21b287d7a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:59:00.105631 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547f7209-bf04-4dad-8319-0bc21b287d7a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:59:00.107704 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547f7209-bf04-4dad-8319-0bc21b287d7a None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:59:00.121766 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application [None req-547f7209-bf04-4dad-8319-0bc21b287d7a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 11:59:00.121766 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 11:59:00.121766 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 11:59:00.121766 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 11:59:00.121766 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 11:59:00.121766 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 11:59:00.121766 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 11:59:00.121766 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 11:59:00.121766 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 11:59:00.121766 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 11:59:00.121766 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 11:59:00.121766 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 11:59:00.121766 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 11:59:00.121766 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 11:59:00.121766 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 11:59:00.122758 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 11:59:00.122758 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 11:59:00.122758 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 11:59:00.122758 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 11:59:00.122758 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 11:59:00.122758 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 11:59:00.122758 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.get_or_create( May 19 11:59:00.122758 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 11:59:00.122758 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application with Lock( May 19 11:59:00.122758 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 11:59:00.122758 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._enter() May 19 11:59:00.122758 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 11:59:00.122758 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 11:59:00.122758 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 11:59:00.122758 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.creator() May 19 11:59:00.122758 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 11:59:00.122758 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application created_value = creator( May 19 11:59:00.123660 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 11:59:00.123660 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 11:59:00.123660 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 11:59:00.123660 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 11:59:00.123660 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 11:59:00.123660 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 11:59:00.123660 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 11:59:00.123660 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application May 19 11:59:00.129469 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 489/987] 10.222.0.23 () {56 vars in 1111 bytes} [Thu May 19 11:58:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 11:59:00.129722 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0631508f-dc1f-4adf-87aa-4954744258ff None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:59:00.155863 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:59:00.241754 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0631508f-dc1f-4adf-87aa-4954744258ff None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:59:00.277296 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a7ad59-71e5-4e36-984c-beeb22869df9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:59:00.277880 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a7ad59-71e5-4e36-984c-beeb22869df9 None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:59:00.278169 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a7ad59-71e5-4e36-984c-beeb22869df9 None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:59:00.278449 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a7ad59-71e5-4e36-984c-beeb22869df9 None admin] PATH_INFO: `/v3/projects` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:59:00.279306 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a7ad59-71e5-4e36-984c-beeb22869df9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 11:59:00.279622 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a7ad59-71e5-4e36-984c-beeb22869df9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:59:00.281238 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a7ad59-71e5-4e36-984c-beeb22869df9 None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:59:00.301905 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 499/988] 10.222.0.23 () {56 vars in 1148 bytes} [Thu May 19 11:59:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:59:00.303499 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547f7209-bf04-4dad-8319-0bc21b287d7a None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:59:00.412043 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547f7209-bf04-4dad-8319-0bc21b287d7a None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:59:00.431859 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:59:00.452148 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe87167-3e58-4e2d-a9a2-e697612e5e59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:59:00.452812 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe87167-3e58-4e2d-a9a2-e697612e5e59 None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:59:00.453096 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe87167-3e58-4e2d-a9a2-e697612e5e59 None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:59:00.461638 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe87167-3e58-4e2d-a9a2-e697612e5e59 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:59:00.461638 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe87167-3e58-4e2d-a9a2-e697612e5e59 None admin] RBAC: Adding query filter params (role.id=ff2dfc5f80304d9cbd3320d8ceffe2a2, scope.project.id=d70d25bf69314820b8aaecb65576978a, user.id=0c3181fb4f584a549a786adfefdf3dfb) {{(pid=80539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 11:59:00.461638 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe87167-3e58-4e2d-a9a2-e697612e5e59 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:59:00.461638 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe87167-3e58-4e2d-a9a2-e697612e5e59 None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:59:00.468190 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 490/989] 10.222.0.23 () {56 vars in 1374 bytes} [Thu May 19 11:59:00 2022] GET /identity/v3/role_assignments?user.id=0c3181fb4f584a549a786adfefdf3dfb&scope.project.id=d70d25bf69314820b8aaecb65576978a&role.id=ff2dfc5f80304d9cbd3320d8ceffe2a2 => generated 608 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 11:59:01.425521 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa433a86-4897-4e03-ab6f-3885aac8972f None None] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:59:01.426578 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa433a86-4897-4e03-ab6f-3885aac8972f None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:59:01.427083 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa433a86-4897-4e03-ab6f-3885aac8972f None None] PATH_INFO: `/` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:59:01.428999 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 500/990] 10.222.0.23 () {54 vars in 926 bytes} [Thu May 19 11:59:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 11:59:01.434167 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b85215-54ed-45d4-8ce6-a0805d23d2e0 None None] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:59:01.435438 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b85215-54ed-45d4-8ce6-a0805d23d2e0 None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:59:01.435974 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b85215-54ed-45d4-8ce6-a0805d23d2e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:59:01.453928 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.auth.core [None req-45b85215-54ed-45d4-8ce6-a0805d23d2e0 None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 11:59:01.455447 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-45b85215-54ed-45d4-8ce6-a0805d23d2e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 11:59:01.526023 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 491/991] 10.222.0.23 () {58 vars in 1023 bytes} [Thu May 19 11:59:01 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 11:59:01.548269 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f9063c-c8b1-4d18-ad60-08ae37606974 None None] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:59:01.548616 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f9063c-c8b1-4d18-ad60-08ae37606974 None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:59:01.548843 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f9063c-c8b1-4d18-ad60-08ae37606974 None None] PATH_INFO: `/` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:59:01.549886 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b85215-54ed-45d4-8ce6-a0805d23d2e0 None None] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:59:01.550343 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 501/992] 10.222.0.23 () {54 vars in 926 bytes} [Thu May 19 11:59:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 11:59:01.555825 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6931db-94f2-423f-9967-f449589ce3dd None None] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:59:01.556155 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6931db-94f2-423f-9967-f449589ce3dd None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:59:01.556445 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6931db-94f2-423f-9967-f449589ce3dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:59:01.575408 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.auth.core [None req-4f6931db-94f2-423f-9967-f449589ce3dd None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 11:59:01.577636 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-4f6931db-94f2-423f-9967-f449589ce3dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 11:59:01.653299 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 502/993] 10.222.0.23 () {58 vars in 1023 bytes} [Thu May 19 11:59:01 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 11:59:01.655156 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b85215-54ed-45d4-8ce6-a0805d23d2e0 None None] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:59:01.666606 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:59:01.677110 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6931db-94f2-423f-9967-f449589ce3dd None None] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:59:01.689297 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:59:01.692719 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d883b204-7845-4240-8f3c-a4887ee0d285 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:59:01.693527 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d883b204-7845-4240-8f3c-a4887ee0d285 None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:59:01.693815 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d883b204-7845-4240-8f3c-a4887ee0d285 None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:59:01.694096 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d883b204-7845-4240-8f3c-a4887ee0d285 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:59:01.694998 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d883b204-7845-4240-8f3c-a4887ee0d285 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:59:01.696867 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d883b204-7845-4240-8f3c-a4887ee0d285 None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:59:01.709833 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application [None req-d883b204-7845-4240-8f3c-a4887ee0d285 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. May 19 11:59:01.709833 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 11:59:01.709833 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 11:59:01.709833 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 11:59:01.709833 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 11:59:01.709833 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 11:59:01.709833 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 11:59:01.709833 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 11:59:01.709833 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 19 11:59:01.709833 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application May 19 11:59:01.709833 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 11:59:01.709833 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application May 19 11:59:01.709833 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 11:59:01.709833 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 11:59:01.709833 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 11:59:01.709833 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 11:59:01.709833 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 11:59:01.710833 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 11:59:01.710833 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 11:59:01.710833 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 11:59:01.710833 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 11:59:01.710833 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 11:59:01.710833 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 11:59:01.710833 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 11:59:01.710833 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 11:59:01.710833 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 11:59:01.710833 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 11:59:01.710833 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 11:59:01.710833 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 11:59:01.710833 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 11:59:01.710833 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 11:59:01.710833 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 11:59:01.710833 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.get_or_create( May 19 11:59:01.711892 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 11:59:01.711892 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application with Lock( May 19 11:59:01.711892 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 11:59:01.711892 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._enter() May 19 11:59:01.711892 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 11:59:01.711892 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 11:59:01.711892 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 11:59:01.711892 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.creator() May 19 11:59:01.711892 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 11:59:01.711892 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application created_value = creator( May 19 11:59:01.711892 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 11:59:01.711892 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 11:59:01.711892 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 19 11:59:01.711892 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application May 19 11:59:01.711892 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 492/994] 10.222.0.23 () {56 vars in 1111 bytes} [Thu May 19 11:59:01 2022] GET /identity/v3/services/compute => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 11:59:01.717467 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d883b204-7845-4240-8f3c-a4887ee0d285 None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:59:01.787622 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6931db-94f2-423f-9967-f449589ce3dd None None] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:59:01.825166 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c096a31-c2a4-40ae-8691-1a49da32e963 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:59:01.826078 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c096a31-c2a4-40ae-8691-1a49da32e963 None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:59:01.826366 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c096a31-c2a4-40ae-8691-1a49da32e963 None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:59:01.826658 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c096a31-c2a4-40ae-8691-1a49da32e963 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:59:01.827321 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c096a31-c2a4-40ae-8691-1a49da32e963 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:59:01.828773 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c096a31-c2a4-40ae-8691-1a49da32e963 None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:59:01.830281 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d883b204-7845-4240-8f3c-a4887ee0d285 None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:59:01.841259 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application [None req-2c096a31-c2a4-40ae-8691-1a49da32e963 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 11:59:01.841259 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 11:59:01.841259 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 11:59:01.841259 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 11:59:01.841259 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 11:59:01.841259 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 11:59:01.841259 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 11:59:01.841259 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 11:59:01.841259 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 11:59:01.841259 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application May 19 11:59:01.841259 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 11:59:01.841259 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application May 19 11:59:01.841259 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 11:59:01.841259 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 11:59:01.841259 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 11:59:01.841259 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 11:59:01.841259 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 11:59:01.842785 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 11:59:01.842785 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 11:59:01.842785 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 11:59:01.842785 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 11:59:01.842785 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 11:59:01.842785 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 11:59:01.842785 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 11:59:01.842785 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 11:59:01.842785 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 11:59:01.842785 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 11:59:01.842785 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 11:59:01.842785 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 11:59:01.842785 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 11:59:01.842785 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 11:59:01.842785 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 11:59:01.842785 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.get_or_create( May 19 11:59:01.843628 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 11:59:01.843628 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application with Lock( May 19 11:59:01.843628 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 11:59:01.843628 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._enter() May 19 11:59:01.843628 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 11:59:01.843628 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 11:59:01.843628 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 11:59:01.843628 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.creator() May 19 11:59:01.843628 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 11:59:01.843628 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application created_value = creator( May 19 11:59:01.843628 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 11:59:01.843628 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 11:59:01.843628 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 11:59:01.843628 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application May 19 11:59:01.843628 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 503/995] 10.222.0.23 () {56 vars in 1126 bytes} [Thu May 19 11:59:01 2022] GET /identity/v3/services/object-store => generated 93 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 11:59:01.848559 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c096a31-c2a4-40ae-8691-1a49da32e963 None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:59:01.870880 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94e6e2e-268a-432a-bb98-d73eb668962b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:59:01.871554 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94e6e2e-268a-432a-bb98-d73eb668962b None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:59:01.871873 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94e6e2e-268a-432a-bb98-d73eb668962b None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:59:01.872162 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94e6e2e-268a-432a-bb98-d73eb668962b None admin] PATH_INFO: `/v3/services` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:59:01.873831 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94e6e2e-268a-432a-bb98-d73eb668962b None admin] RBAC: Adding query filter params (name=compute) {{(pid=80539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 11:59:01.874241 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94e6e2e-268a-432a-bb98-d73eb668962b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:59:01.875541 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94e6e2e-268a-432a-bb98-d73eb668962b None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:59:01.881014 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 493/996] 10.222.0.23 () {56 vars in 1112 bytes} [Thu May 19 11:59:01 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 11:59:01.886502 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94e6e2e-268a-432a-bb98-d73eb668962b None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:59:01.926517 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:59:01.941137 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:59:01.970818 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c096a31-c2a4-40ae-8691-1a49da32e963 None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:59:01.996460 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94e6e2e-268a-432a-bb98-d73eb668962b None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:59:02.011744 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4952f493-3aab-41f8-970c-0e1cdf3eec5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:59:02.012527 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4952f493-3aab-41f8-970c-0e1cdf3eec5b None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:59:02.012813 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4952f493-3aab-41f8-970c-0e1cdf3eec5b None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:59:02.013116 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4952f493-3aab-41f8-970c-0e1cdf3eec5b None admin] PATH_INFO: `/v3/services` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:59:02.014011 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4952f493-3aab-41f8-970c-0e1cdf3eec5b None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 11:59:02.014340 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4952f493-3aab-41f8-970c-0e1cdf3eec5b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:59:02.015725 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4952f493-3aab-41f8-970c-0e1cdf3eec5b None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:59:02.025345 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 504/997] 10.222.0.23 () {56 vars in 1122 bytes} [Thu May 19 11:59:01 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 11:59:02.026738 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4952f493-3aab-41f8-970c-0e1cdf3eec5b None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:59:02.040933 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7605f0-af95-47a1-b061-a17335f2298d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:59:02.041616 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7605f0-af95-47a1-b061-a17335f2298d None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:59:02.041906 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7605f0-af95-47a1-b061-a17335f2298d None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:59:02.042359 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7605f0-af95-47a1-b061-a17335f2298d None admin] PATH_INFO: `/v3/services` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:59:02.043509 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7605f0-af95-47a1-b061-a17335f2298d None admin] RBAC: Adding query filter params (type=compute) {{(pid=80539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 11:59:02.043858 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7605f0-af95-47a1-b061-a17335f2298d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:59:02.046145 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7605f0-af95-47a1-b061-a17335f2298d None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:59:02.051663 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 494/998] 10.222.0.23 () {56 vars in 1112 bytes} [Thu May 19 11:59:01 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 11:59:02.057382 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7605f0-af95-47a1-b061-a17335f2298d None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:59:02.144324 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4952f493-3aab-41f8-970c-0e1cdf3eec5b None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:59:02.172137 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7605f0-af95-47a1-b061-a17335f2298d None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:59:02.173941 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:59:02.182255 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9aab432-c738-4984-8b99-8a606566cb89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:59:02.182935 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9aab432-c738-4984-8b99-8a606566cb89 None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:59:02.183217 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9aab432-c738-4984-8b99-8a606566cb89 None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:59:02.183497 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9aab432-c738-4984-8b99-8a606566cb89 None admin] PATH_INFO: `/v3/services` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:59:02.184237 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9aab432-c738-4984-8b99-8a606566cb89 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 11:59:02.184544 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9aab432-c738-4984-8b99-8a606566cb89 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:59:02.186479 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9aab432-c738-4984-8b99-8a606566cb89 None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:59:02.189141 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:59:02.198769 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 505/999] 10.222.0.23 () {56 vars in 1122 bytes} [Thu May 19 11:59:02 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 11:59:02.199940 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9aab432-c738-4984-8b99-8a606566cb89 None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:59:02.227756 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916a105d-4108-4dab-897e-ea69599b72e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:59:02.228442 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916a105d-4108-4dab-897e-ea69599b72e4 None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:59:02.228725 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916a105d-4108-4dab-897e-ea69599b72e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:59:02.229225 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916a105d-4108-4dab-897e-ea69599b72e4 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:59:02.229863 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916a105d-4108-4dab-897e-ea69599b72e4 None admin] RBAC: Adding query filter params (interface=public, service_id=dbc9c420aedf43edbf32120276f6fa47, region_id=RegionOne) {{(pid=80539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 11:59:02.230175 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916a105d-4108-4dab-897e-ea69599b72e4 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:59:02.231562 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916a105d-4108-4dab-897e-ea69599b72e4 None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:59:02.238111 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 495/1000] 10.222.0.23 () {56 vars in 1251 bytes} [Thu May 19 11:59:02 2022] GET /identity/v3/endpoints?service_id=dbc9c420aedf43edbf32120276f6fa47&interface=public®ion_id=RegionOne => generated 194 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:59:02.243713 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916a105d-4108-4dab-897e-ea69599b72e4 None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:59:02.318867 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9aab432-c738-4984-8b99-8a606566cb89 None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:59:02.353127 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916a105d-4108-4dab-897e-ea69599b72e4 None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:59:02.355476 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138e6f1c-b081-459b-93ec-bd88bb6449db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:59:02.356170 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138e6f1c-b081-459b-93ec-bd88bb6449db None admin] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:59:02.356455 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138e6f1c-b081-459b-93ec-bd88bb6449db None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:59:02.356737 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138e6f1c-b081-459b-93ec-bd88bb6449db None admin] PATH_INFO: `/v3/endpoints` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:59:02.358133 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138e6f1c-b081-459b-93ec-bd88bb6449db None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:59:02.359517 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138e6f1c-b081-459b-93ec-bd88bb6449db None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:59:02.389172 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4ea696-7eb4-45b6-8530-52bd6b18ae42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:59:02.389834 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4ea696-7eb4-45b6-8530-52bd6b18ae42 None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:59:02.390122 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4ea696-7eb4-45b6-8530-52bd6b18ae42 None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:59:02.390401 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4ea696-7eb4-45b6-8530-52bd6b18ae42 None admin] PATH_INFO: `/v3/services` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:59:02.391085 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4ea696-7eb4-45b6-8530-52bd6b18ae42 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:59:02.392494 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4ea696-7eb4-45b6-8530-52bd6b18ae42 None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:59:02.398761 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 496/1001] 10.222.0.23 () {56 vars in 1087 bytes} [Thu May 19 11:59:02 2022] GET /identity/v3/services => generated 1727 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 11:59:02.456759 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 506/1002] 10.222.0.23 () {60 vars in 1144 bytes} [Thu May 19 11:59:02 2022] POST /identity/v3/endpoints => generated 325 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 11:59:02.760240 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d959a6-7751-44f3-b5c4-e7ee22ba955c None None] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:59:02.760726 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d959a6-7751-44f3-b5c4-e7ee22ba955c None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:59:02.760726 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d959a6-7751-44f3-b5c4-e7ee22ba955c None None] PATH_INFO: `/` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:59:02.762142 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 497/1003] 10.222.0.23 () {54 vars in 926 bytes} [Thu May 19 11:59:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 11:59:02.768313 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0dc0aae-5d5f-4182-a867-90b6c5df45d2 None None] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:59:02.768620 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0dc0aae-5d5f-4182-a867-90b6c5df45d2 None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:59:02.768910 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0dc0aae-5d5f-4182-a867-90b6c5df45d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:59:02.786809 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.auth.core [None req-a0dc0aae-5d5f-4182-a867-90b6c5df45d2 None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 11:59:02.788060 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-a0dc0aae-5d5f-4182-a867-90b6c5df45d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 11:59:02.846894 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:59:02.864800 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 507/1004] 10.222.0.23 () {58 vars in 1023 bytes} [Thu May 19 11:59:02 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 11:59:02.888775 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d959a6-7751-44f3-b5c4-e7ee22ba955c None None] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:59:02.923538 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:59:03.006490 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d959a6-7751-44f3-b5c4-e7ee22ba955c None None] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:59:03.042528 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf7b935-1e27-4b43-9fe3-8d7484e60fc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:59:03.043648 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf7b935-1e27-4b43-9fe3-8d7484e60fc0 None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:59:03.044237 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf7b935-1e27-4b43-9fe3-8d7484e60fc0 None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:59:03.044718 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf7b935-1e27-4b43-9fe3-8d7484e60fc0 None admin] PATH_INFO: `/v3/services/image` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:59:03.045693 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf7b935-1e27-4b43-9fe3-8d7484e60fc0 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:59:03.047268 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf7b935-1e27-4b43-9fe3-8d7484e60fc0 None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:59:03.059496 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application [None req-2bf7b935-1e27-4b43-9fe3-8d7484e60fc0 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. May 19 11:59:03.059496 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 11:59:03.059496 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 11:59:03.059496 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 11:59:03.059496 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 11:59:03.059496 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 11:59:03.059496 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 11:59:03.059496 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 11:59:03.059496 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 19 11:59:03.059496 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application May 19 11:59:03.059496 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 11:59:03.059496 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application May 19 11:59:03.059496 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 11:59:03.059496 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 11:59:03.059496 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 11:59:03.059496 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 11:59:03.059496 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 11:59:03.061335 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 11:59:03.061335 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 11:59:03.061335 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 11:59:03.061335 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 11:59:03.061335 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 11:59:03.061335 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 11:59:03.061335 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 11:59:03.061335 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 11:59:03.061335 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 11:59:03.061335 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 11:59:03.061335 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 11:59:03.061335 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 11:59:03.061335 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 11:59:03.061335 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 11:59:03.061335 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 11:59:03.061335 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.get_or_create( May 19 11:59:03.062402 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 11:59:03.062402 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application with Lock( May 19 11:59:03.062402 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 11:59:03.062402 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._enter() May 19 11:59:03.062402 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 11:59:03.062402 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 11:59:03.062402 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 11:59:03.062402 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.creator() May 19 11:59:03.062402 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 11:59:03.062402 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application created_value = creator( May 19 11:59:03.062402 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 11:59:03.062402 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 11:59:03.062402 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 19 11:59:03.062402 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application May 19 11:59:03.065076 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 498/1005] 10.222.0.23 () {56 vars in 1105 bytes} [Thu May 19 11:59:02 2022] GET /identity/v3/services/image => generated 86 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 11:59:03.071354 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0dc0aae-5d5f-4182-a867-90b6c5df45d2 None None] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:59:03.173306 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0dc0aae-5d5f-4182-a867-90b6c5df45d2 None None] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:59:03.209344 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9596a87e-60f6-4890-8067-aa4a3c3560b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:59:03.210072 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9596a87e-60f6-4890-8067-aa4a3c3560b7 None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:59:03.210362 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9596a87e-60f6-4890-8067-aa4a3c3560b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:59:03.210638 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9596a87e-60f6-4890-8067-aa4a3c3560b7 None admin] PATH_INFO: `/v3/services` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:59:03.211352 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9596a87e-60f6-4890-8067-aa4a3c3560b7 None admin] RBAC: Adding query filter params (name=image) {{(pid=80540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 11:59:03.211663 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9596a87e-60f6-4890-8067-aa4a3c3560b7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:59:03.213084 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9596a87e-60f6-4890-8067-aa4a3c3560b7 None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:59:03.222830 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 508/1006] 10.222.0.23 () {56 vars in 1108 bytes} [Thu May 19 11:59:03 2022] GET /identity/v3/services?name=image => generated 122 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:59:03.224170 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf7b935-1e27-4b43-9fe3-8d7484e60fc0 None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:59:03.328642 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:59:03.335851 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf7b935-1e27-4b43-9fe3-8d7484e60fc0 None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:59:03.370020 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817e990f-6bb6-41b3-bb72-fcc45dc634de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:59:03.370685 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817e990f-6bb6-41b3-bb72-fcc45dc634de None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:59:03.370973 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817e990f-6bb6-41b3-bb72-fcc45dc634de None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:59:03.371251 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817e990f-6bb6-41b3-bb72-fcc45dc634de None admin] PATH_INFO: `/v3/services` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:59:03.371987 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-817e990f-6bb6-41b3-bb72-fcc45dc634de None admin] RBAC: Adding query filter params (type=image) {{(pid=80539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 11:59:03.372298 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-817e990f-6bb6-41b3-bb72-fcc45dc634de None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:59:03.374270 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-817e990f-6bb6-41b3-bb72-fcc45dc634de None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:59:03.387892 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 499/1007] 10.222.0.23 () {56 vars in 1108 bytes} [Thu May 19 11:59:03 2022] GET /identity/v3/services?type=image => generated 122 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:59:04.701218 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fffdf6c-c6f9-4e5a-b4d8-d347c1fe9693 None None] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:59:04.702079 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fffdf6c-c6f9-4e5a-b4d8-d347c1fe9693 None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:59:04.702079 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fffdf6c-c6f9-4e5a-b4d8-d347c1fe9693 None None] PATH_INFO: `/` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:59:04.703565 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 509/1008] 10.222.0.23 () {54 vars in 926 bytes} [Thu May 19 11:59:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 11:59:04.710152 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809e4749-2141-4a9c-be1a-35f8906e688a None None] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:59:04.710727 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809e4749-2141-4a9c-be1a-35f8906e688a None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:59:04.710727 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809e4749-2141-4a9c-be1a-35f8906e688a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:59:04.727921 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.auth.core [None req-809e4749-2141-4a9c-be1a-35f8906e688a None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 11:59:04.729106 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-809e4749-2141-4a9c-be1a-35f8906e688a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 11:59:04.749635 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9da4e36-1123-446d-b6f2-141fa3dd5758 None None] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:59:04.749881 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9da4e36-1123-446d-b6f2-141fa3dd5758 None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:59:04.750152 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9da4e36-1123-446d-b6f2-141fa3dd5758 None None] PATH_INFO: `/` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:59:04.751736 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 510/1009] 10.222.0.23 () {54 vars in 926 bytes} [Thu May 19 11:59:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 11:59:04.757084 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be112f3-e12b-4fad-ac61-bc08bdbe8a8c None None] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:59:04.757473 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be112f3-e12b-4fad-ac61-bc08bdbe8a8c None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:59:04.757824 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be112f3-e12b-4fad-ac61-bc08bdbe8a8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:59:04.776817 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.auth.core [None req-5be112f3-e12b-4fad-ac61-bc08bdbe8a8c None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 11:59:04.783479 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-5be112f3-e12b-4fad-ac61-bc08bdbe8a8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 11:59:04.806266 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 500/1010] 10.222.0.23 () {58 vars in 1023 bytes} [Thu May 19 11:59:04 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 11:59:04.829560 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809e4749-2141-4a9c-be1a-35f8906e688a None None] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:59:04.830764 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:59:04.853501 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 511/1011] 10.222.0.23 () {58 vars in 1023 bytes} [Thu May 19 11:59:04 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 11:59:04.877489 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be112f3-e12b-4fad-ac61-bc08bdbe8a8c None None] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:59:04.963485 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:59:04.975943 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809e4749-2141-4a9c-be1a-35f8906e688a None None] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:59:05.009745 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be112f3-e12b-4fad-ac61-bc08bdbe8a8c None None] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:59:05.012594 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cecd608-f0c8-4bc5-9131-0446ef3a0ef9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:59:05.013985 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cecd608-f0c8-4bc5-9131-0446ef3a0ef9 None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:59:05.014281 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cecd608-f0c8-4bc5-9131-0446ef3a0ef9 None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:59:05.014625 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cecd608-f0c8-4bc5-9131-0446ef3a0ef9 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:59:05.015276 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cecd608-f0c8-4bc5-9131-0446ef3a0ef9 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:59:05.016674 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cecd608-f0c8-4bc5-9131-0446ef3a0ef9 None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:59:05.029359 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application [None req-8cecd608-f0c8-4bc5-9131-0446ef3a0ef9 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. May 19 11:59:05.029359 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 11:59:05.029359 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 11:59:05.029359 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 11:59:05.029359 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 11:59:05.029359 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 11:59:05.029359 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 11:59:05.029359 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 11:59:05.029359 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 19 11:59:05.029359 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application May 19 11:59:05.029359 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 11:59:05.029359 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application May 19 11:59:05.029359 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 11:59:05.029359 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 11:59:05.029359 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 11:59:05.029359 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 11:59:05.029359 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 11:59:05.030696 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 11:59:05.030696 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 11:59:05.030696 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 11:59:05.030696 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 11:59:05.030696 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 11:59:05.030696 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 11:59:05.030696 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 11:59:05.030696 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 11:59:05.030696 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 11:59:05.030696 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 11:59:05.030696 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 11:59:05.030696 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 11:59:05.030696 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 11:59:05.030696 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 11:59:05.030696 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 11:59:05.030696 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.get_or_create( May 19 11:59:05.031599 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 11:59:05.031599 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application with Lock( May 19 11:59:05.031599 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 11:59:05.031599 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._enter() May 19 11:59:05.031599 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 11:59:05.031599 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 11:59:05.031599 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 11:59:05.031599 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.creator() May 19 11:59:05.031599 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 11:59:05.031599 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application created_value = creator( May 19 11:59:05.031599 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 11:59:05.031599 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 11:59:05.031599 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 19 11:59:05.031599 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application May 19 11:59:05.031599 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 501/1012] 10.222.0.23 () {56 vars in 1111 bytes} [Thu May 19 11:59:04 2022] GET /identity/v3/services/compute => generated 88 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 11:59:05.036824 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cecd608-f0c8-4bc5-9131-0446ef3a0ef9 None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:59:05.052591 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b6e258-1eaf-4fee-93c4-249f920cee31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:59:05.053817 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b6e258-1eaf-4fee-93c4-249f920cee31 None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:59:05.054332 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b6e258-1eaf-4fee-93c4-249f920cee31 None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:59:05.054792 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b6e258-1eaf-4fee-93c4-249f920cee31 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:59:05.056459 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b6e258-1eaf-4fee-93c4-249f920cee31 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:59:05.058114 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b6e258-1eaf-4fee-93c4-249f920cee31 None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:59:05.060977 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 512/1013] 10.222.0.23 () {56 vars in 1108 bytes} [Thu May 19 11:59:04 2022] GET /identity/v3/domains/default => generated 199 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 11:59:05.067000 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b6e258-1eaf-4fee-93c4-249f920cee31 None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:59:05.123551 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:59:05.151376 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cecd608-f0c8-4bc5-9131-0446ef3a0ef9 None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:59:05.177411 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b6e258-1eaf-4fee-93c4-249f920cee31 None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:59:05.196619 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f550f620-f240-42f2-931e-cbb0dedc7a9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:59:05.197288 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f550f620-f240-42f2-931e-cbb0dedc7a9a None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:59:05.197569 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f550f620-f240-42f2-931e-cbb0dedc7a9a None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:59:05.197846 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f550f620-f240-42f2-931e-cbb0dedc7a9a None admin] PATH_INFO: `/v3/services` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:59:05.198995 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f550f620-f240-42f2-931e-cbb0dedc7a9a None admin] RBAC: Adding query filter params (name=compute) {{(pid=80539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 11:59:05.199452 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f550f620-f240-42f2-931e-cbb0dedc7a9a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:59:05.201686 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f550f620-f240-42f2-931e-cbb0dedc7a9a None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:59:05.212994 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 502/1014] 10.222.0.23 () {56 vars in 1112 bytes} [Thu May 19 11:59:05 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 11:59:05.213700 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3773904-655d-41f7-8f26-9637c017e83d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:59:05.214087 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f550f620-f240-42f2-931e-cbb0dedc7a9a None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:59:05.214366 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3773904-655d-41f7-8f26-9637c017e83d None admin] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:59:05.214722 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3773904-655d-41f7-8f26-9637c017e83d None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:59:05.215004 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3773904-655d-41f7-8f26-9637c017e83d None admin] PATH_INFO: `/v3/projects` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:59:05.215721 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3773904-655d-41f7-8f26-9637c017e83d None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:59:05.218086 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3773904-655d-41f7-8f26-9637c017e83d None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:59:05.234632 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:59:05.288164 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 513/1015] 10.222.0.23 () {60 vars in 1141 bytes} [Thu May 19 11:59:05 2022] POST /identity/v3/projects => generated 311 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 11:59:05.327048 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f550f620-f240-42f2-931e-cbb0dedc7a9a None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:59:05.367379 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ecfe37c-f363-4b57-9999-abe47e0179db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:59:05.368105 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecfe37c-f363-4b57-9999-abe47e0179db None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:59:05.368389 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecfe37c-f363-4b57-9999-abe47e0179db None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:59:05.368664 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecfe37c-f363-4b57-9999-abe47e0179db None admin] PATH_INFO: `/v3/services` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:59:05.369393 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ecfe37c-f363-4b57-9999-abe47e0179db None admin] RBAC: Adding query filter params (type=compute) {{(pid=80539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 11:59:05.369696 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ecfe37c-f363-4b57-9999-abe47e0179db None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:59:05.371104 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ecfe37c-f363-4b57-9999-abe47e0179db None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:59:05.381712 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 503/1016] 10.222.0.23 () {56 vars in 1112 bytes} [Thu May 19 11:59:05 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:59:05.382776 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3773904-655d-41f7-8f26-9637c017e83d None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:59:05.495770 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:59:05.499091 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3773904-655d-41f7-8f26-9637c017e83d None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:59:05.541335 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3fc65d-f68b-4d1b-8524-911641c59c93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:59:05.542070 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3fc65d-f68b-4d1b-8524-911641c59c93 None admin] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:59:05.542292 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3fc65d-f68b-4d1b-8524-911641c59c93 None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:59:05.542569 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3fc65d-f68b-4d1b-8524-911641c59c93 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:59:05.543322 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3fc65d-f68b-4d1b-8524-911641c59c93 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:59:05.544929 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3fc65d-f68b-4d1b-8524-911641c59c93 None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:59:05.639758 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 514/1017] 10.222.0.23 () {60 vars in 1144 bytes} [Thu May 19 11:59:05 2022] POST /identity/v3/endpoints => generated 334 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 11:59:05.652594 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cf798e-85e6-4b7e-952c-4d0aa27aae95 None None] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:59:05.653084 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cf798e-85e6-4b7e-952c-4d0aa27aae95 None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:59:05.653438 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cf798e-85e6-4b7e-952c-4d0aa27aae95 None None] PATH_INFO: `/` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:59:05.656414 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 504/1018] 10.222.0.23 () {54 vars in 926 bytes} [Thu May 19 11:59:05 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 11:59:05.662326 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c13e81c-491f-4933-a025-0786ab47a608 None None] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:59:05.662653 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c13e81c-491f-4933-a025-0786ab47a608 None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:59:05.662914 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c13e81c-491f-4933-a025-0786ab47a608 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:59:05.681331 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.auth.core [None req-7c13e81c-491f-4933-a025-0786ab47a608 None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 11:59:05.682582 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-7c13e81c-491f-4933-a025-0786ab47a608 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 11:59:05.781459 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 515/1019] 10.222.0.23 () {58 vars in 1023 bytes} [Thu May 19 11:59:05 2022] POST /identity/v3/auth/tokens => generated 1863 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 11:59:05.804887 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84cf798e-85e6-4b7e-952c-4d0aa27aae95 None None] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:59:05.897995 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:59:05.918936 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84cf798e-85e6-4b7e-952c-4d0aa27aae95 None None] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:59:05.966406 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3e7709-0f85-42b3-9100-bd5aba931fdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:59:05.967626 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3e7709-0f85-42b3-9100-bd5aba931fdf None admin] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:59:05.967996 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3e7709-0f85-42b3-9100-bd5aba931fdf None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:59:05.968276 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3e7709-0f85-42b3-9100-bd5aba931fdf None admin] PATH_INFO: `/v3/services` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:59:05.969009 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3e7709-0f85-42b3-9100-bd5aba931fdf None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:59:05.970396 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3e7709-0f85-42b3-9100-bd5aba931fdf None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:59:06.063112 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 505/1020] 10.222.0.23 () {60 vars in 1141 bytes} [Thu May 19 11:59:05 2022] POST /identity/v3/services => generated 242 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 11:59:07.620684 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a987dd-15b9-4e95-bf6b-61ef0a3687d8 None None] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:59:07.621229 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a987dd-15b9-4e95-bf6b-61ef0a3687d8 None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:59:07.621229 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a987dd-15b9-4e95-bf6b-61ef0a3687d8 None None] PATH_INFO: `/` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:59:07.622715 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 516/1021] 10.222.0.23 () {54 vars in 926 bytes} [Thu May 19 11:59:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 11:59:07.628005 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec71f51-8815-4a8c-97c0-e0588c439000 None None] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:59:07.628403 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec71f51-8815-4a8c-97c0-e0588c439000 None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:59:07.628570 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec71f51-8815-4a8c-97c0-e0588c439000 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:59:07.645737 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.auth.core [None req-8ec71f51-8815-4a8c-97c0-e0588c439000 None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 11:59:07.646937 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-8ec71f51-8815-4a8c-97c0-e0588c439000 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 11:59:07.725808 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 506/1022] 10.222.0.23 () {58 vars in 1023 bytes} [Thu May 19 11:59:07 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 11:59:07.752652 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a987dd-15b9-4e95-bf6b-61ef0a3687d8 None None] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:59:07.842931 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:59:07.864504 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a987dd-15b9-4e95-bf6b-61ef0a3687d8 None None] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:59:07.904958 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd25192-7822-4d04-bb11-4a97a9173627 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:59:07.905687 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd25192-7822-4d04-bb11-4a97a9173627 None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:59:07.906013 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd25192-7822-4d04-bb11-4a97a9173627 None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:59:07.906279 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd25192-7822-4d04-bb11-4a97a9173627 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:59:07.907985 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd25192-7822-4d04-bb11-4a97a9173627 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:59:07.910033 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd25192-7822-4d04-bb11-4a97a9173627 None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:59:07.916220 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 517/1023] 10.222.0.23 () {56 vars in 1108 bytes} [Thu May 19 11:59:07 2022] GET /identity/v3/domains/default => generated 199 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:59:07.917538 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec71f51-8815-4a8c-97c0-e0588c439000 None None] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:59:08.003269 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:59:08.047653 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec71f51-8815-4a8c-97c0-e0588c439000 None None] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:59:08.065243 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f21ddf-32d0-453c-81a5-cd4c53fde055 None None] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:59:08.065540 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f21ddf-32d0-453c-81a5-cd4c53fde055 None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:59:08.065871 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f21ddf-32d0-453c-81a5-cd4c53fde055 None None] PATH_INFO: `/` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:59:08.067522 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 518/1024] 10.222.0.23 () {54 vars in 926 bytes} [Thu May 19 11:59:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 11:59:08.073083 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913b3884-3b6b-490e-8b7b-2ac433d17cf9 None None] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:59:08.073673 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913b3884-3b6b-490e-8b7b-2ac433d17cf9 None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:59:08.073963 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913b3884-3b6b-490e-8b7b-2ac433d17cf9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:59:08.086490 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b7dbc7-a070-4fb5-bc8e-8cb0ca4aef8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:59:08.087195 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b7dbc7-a070-4fb5-bc8e-8cb0ca4aef8f None admin] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:59:08.087450 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b7dbc7-a070-4fb5-bc8e-8cb0ca4aef8f None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:59:08.087721 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b7dbc7-a070-4fb5-bc8e-8cb0ca4aef8f None admin] PATH_INFO: `/v3/users` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:59:08.088478 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b7dbc7-a070-4fb5-bc8e-8cb0ca4aef8f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:59:08.089906 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b7dbc7-a070-4fb5-bc8e-8cb0ca4aef8f None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:59:08.092361 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.auth.core [None req-913b3884-3b6b-490e-8b7b-2ac433d17cf9 None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 11:59:08.093570 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-913b3884-3b6b-490e-8b7b-2ac433d17cf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 11:59:08.161923 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 519/1025] 10.222.0.23 () {58 vars in 1023 bytes} [Thu May 19 11:59:08 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 11:59:08.186739 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913b3884-3b6b-490e-8b7b-2ac433d17cf9 None None] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:59:08.203826 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 507/1026] 10.222.0.23 () {60 vars in 1132 bytes} [Thu May 19 11:59:07 2022] POST /identity/v3/users => generated 285 bytes in 278 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 11:59:08.297004 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:59:08.319172 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913b3884-3b6b-490e-8b7b-2ac433d17cf9 None None] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:59:08.359682 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69791c7-6829-4f7c-931d-cba40cd23a27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:59:08.360576 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69791c7-6829-4f7c-931d-cba40cd23a27 None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:59:08.360864 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69791c7-6829-4f7c-931d-cba40cd23a27 None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:59:08.361136 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69791c7-6829-4f7c-931d-cba40cd23a27 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:59:08.372907 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69791c7-6829-4f7c-931d-cba40cd23a27 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 11:59:08.381478 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application [None req-d69791c7-6829-4f7c-931d-cba40cd23a27 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 19 11:59:08.381478 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 11:59:08.381478 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 11:59:08.381478 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 11:59:08.381478 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 11:59:08.381478 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 11:59:08.381478 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 11:59:08.381478 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 11:59:08.381478 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 11:59:08.381478 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 11:59:08.381478 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 11:59:08.381478 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 11:59:08.381478 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 11:59:08.381478 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 11:59:08.381478 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 11:59:08.381478 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 11:59:08.382488 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 11:59:08.382488 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 11:59:08.382488 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 11:59:08.382488 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 11:59:08.382488 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 11:59:08.382488 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 11:59:08.382488 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 11:59:08.382488 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 11:59:08.382488 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 11:59:08.382488 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.get_or_create( May 19 11:59:08.382488 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 11:59:08.382488 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application with Lock( May 19 11:59:08.382488 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 11:59:08.382488 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._enter() May 19 11:59:08.382488 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 11:59:08.382488 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 11:59:08.383337 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 11:59:08.383337 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.creator() May 19 11:59:08.383337 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 11:59:08.383337 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application created_value = creator( May 19 11:59:08.383337 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 11:59:08.383337 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 11:59:08.383337 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 11:59:08.383337 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 11:59:08.383337 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 11:59:08.383337 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 11:59:08.383337 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 19 11:59:08.383337 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application May 19 11:59:08.387816 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 520/1027] 10.222.0.23 () {56 vars in 1120 bytes} [Thu May 19 11:59:08 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 11:59:08.389257 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b7dbc7-a070-4fb5-bc8e-8cb0ca4aef8f None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:59:08.421431 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd97bbb8-76f9-4f01-9475-19d4b9b65976 None None] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:59:08.421727 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd97bbb8-76f9-4f01-9475-19d4b9b65976 None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:59:08.421975 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd97bbb8-76f9-4f01-9475-19d4b9b65976 None None] PATH_INFO: `/` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:59:08.423367 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 521/1028] 10.222.0.23 () {54 vars in 926 bytes} [Thu May 19 11:59:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 11:59:08.429258 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8817daed-5084-4ba4-a67a-daa5531138e3 None None] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:59:08.429550 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8817daed-5084-4ba4-a67a-daa5531138e3 None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:59:08.429826 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8817daed-5084-4ba4-a67a-daa5531138e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:59:08.447091 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.auth.core [None req-8817daed-5084-4ba4-a67a-daa5531138e3 None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 11:59:08.448399 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-8817daed-5084-4ba4-a67a-daa5531138e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 11:59:08.508060 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b7dbc7-a070-4fb5-bc8e-8cb0ca4aef8f None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:59:08.526688 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 522/1029] 10.222.0.23 () {58 vars in 1023 bytes} [Thu May 19 11:59:08 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 11:59:08.535165 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:59:08.547482 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7172355e-e830-480c-9b29-12b439d023b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:59:08.548198 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7172355e-e830-480c-9b29-12b439d023b0 None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:59:08.562572 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7172355e-e830-480c-9b29-12b439d023b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:59:08.562572 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7172355e-e830-480c-9b29-12b439d023b0 None admin] PATH_INFO: `/v3/roles` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:59:08.562572 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7172355e-e830-480c-9b29-12b439d023b0 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 11:59:08.562572 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7172355e-e830-480c-9b29-12b439d023b0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:59:08.562934 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8817daed-5084-4ba4-a67a-daa5531138e3 None None] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:59:08.563018 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7172355e-e830-480c-9b29-12b439d023b0 None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:59:08.567957 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 508/1030] 10.222.0.23 () {56 vars in 1115 bytes} [Thu May 19 11:59:08 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:59:08.569187 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7172355e-e830-480c-9b29-12b439d023b0 None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:59:08.627002 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:59:08.660696 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8817daed-5084-4ba4-a67a-daa5531138e3 None None] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:59:08.680266 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7172355e-e830-480c-9b29-12b439d023b0 None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:59:08.703016 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2165b0f0-9d82-4d59-ae1d-84931d8898ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:59:08.703845 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2165b0f0-9d82-4d59-ae1d-84931d8898ae None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:59:08.704136 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2165b0f0-9d82-4d59-ae1d-84931d8898ae None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:59:08.704423 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2165b0f0-9d82-4d59-ae1d-84931d8898ae None admin] PATH_INFO: `/v3/services/image` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:59:08.705169 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2165b0f0-9d82-4d59-ae1d-84931d8898ae None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:59:08.706580 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2165b0f0-9d82-4d59-ae1d-84931d8898ae None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:59:08.718554 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application [None req-2165b0f0-9d82-4d59-ae1d-84931d8898ae None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. May 19 11:59:08.718554 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 11:59:08.718554 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 11:59:08.718554 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 11:59:08.718554 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 11:59:08.718554 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 11:59:08.718554 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 11:59:08.718554 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 11:59:08.718554 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 19 11:59:08.718554 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application May 19 11:59:08.718554 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 11:59:08.718554 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application May 19 11:59:08.718554 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 11:59:08.718554 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 11:59:08.718554 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 11:59:08.718554 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 11:59:08.718554 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 11:59:08.719599 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 11:59:08.719599 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 11:59:08.719599 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 11:59:08.719599 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 11:59:08.719599 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 11:59:08.719599 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 11:59:08.719599 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 11:59:08.719599 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 11:59:08.719599 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 11:59:08.719599 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 11:59:08.719599 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 11:59:08.719599 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 11:59:08.719599 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 11:59:08.719599 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 11:59:08.719599 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 11:59:08.719599 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.get_or_create( May 19 11:59:08.720493 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 11:59:08.720493 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application with Lock( May 19 11:59:08.720493 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 11:59:08.720493 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._enter() May 19 11:59:08.720493 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 11:59:08.720493 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 11:59:08.720493 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 11:59:08.720493 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.creator() May 19 11:59:08.720493 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 11:59:08.720493 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application created_value = creator( May 19 11:59:08.720493 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 11:59:08.720493 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 11:59:08.720493 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 19 11:59:08.720493 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application May 19 11:59:08.720493 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 523/1031] 10.222.0.23 () {56 vars in 1105 bytes} [Thu May 19 11:59:08 2022] GET /identity/v3/services/image => generated 86 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 11:59:08.722216 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54cf71a-3a0d-48dc-ab93-e0c71d289928 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:59:08.723088 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54cf71a-3a0d-48dc-ab93-e0c71d289928 None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:59:08.723380 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54cf71a-3a0d-48dc-ab93-e0c71d289928 None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:59:08.723658 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54cf71a-3a0d-48dc-ab93-e0c71d289928 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:59:08.725670 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2165b0f0-9d82-4d59-ae1d-84931d8898ae None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:59:08.735486 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54cf71a-3a0d-48dc-ab93-e0c71d289928 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:59:08.736997 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54cf71a-3a0d-48dc-ab93-e0c71d289928 None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:59:08.757141 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application [None req-e54cf71a-3a0d-48dc-ab93-e0c71d289928 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 11:59:08.757141 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 11:59:08.757141 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 11:59:08.757141 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 11:59:08.757141 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 11:59:08.757141 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 11:59:08.757141 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 11:59:08.757141 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 11:59:08.757141 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 11:59:08.757141 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 11:59:08.757141 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 11:59:08.757141 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 11:59:08.757141 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 11:59:08.757141 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 11:59:08.757141 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 11:59:08.758845 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 11:59:08.758845 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 11:59:08.758845 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 11:59:08.758845 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 11:59:08.758845 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 11:59:08.758845 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 11:59:08.758845 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.get_or_create( May 19 11:59:08.758845 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 11:59:08.758845 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application with Lock( May 19 11:59:08.758845 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 11:59:08.758845 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._enter() May 19 11:59:08.758845 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 11:59:08.758845 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 11:59:08.758845 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 11:59:08.758845 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.creator() May 19 11:59:08.758845 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 11:59:08.758845 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application created_value = creator( May 19 11:59:08.760643 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 11:59:08.760643 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 11:59:08.760643 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 11:59:08.760643 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application May 19 11:59:08.760643 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 509/1032] 10.222.0.23 () {56 vars in 1108 bytes} [Thu May 19 11:59:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 11:59:08.764883 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54cf71a-3a0d-48dc-ab93-e0c71d289928 None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:59:08.803495 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:59:08.845329 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2165b0f0-9d82-4d59-ae1d-84931d8898ae None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:59:08.878406 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54cf71a-3a0d-48dc-ab93-e0c71d289928 None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:59:08.880820 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e712537-8d23-41c0-bc95-6829d706d0f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:59:08.881507 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e712537-8d23-41c0-bc95-6829d706d0f9 None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:59:08.881792 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e712537-8d23-41c0-bc95-6829d706d0f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:59:08.882070 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e712537-8d23-41c0-bc95-6829d706d0f9 None admin] PATH_INFO: `/v3/services` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:59:08.882789 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e712537-8d23-41c0-bc95-6829d706d0f9 None admin] RBAC: Adding query filter params (name=image) {{(pid=80540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 11:59:08.883103 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e712537-8d23-41c0-bc95-6829d706d0f9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:59:08.884530 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e712537-8d23-41c0-bc95-6829d706d0f9 None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:59:08.885774 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:59:08.891298 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 524/1033] 10.222.0.23 () {56 vars in 1108 bytes} [Thu May 19 11:59:08 2022] GET /identity/v3/services?name=image => generated 122 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:59:08.900489 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e712537-8d23-41c0-bc95-6829d706d0f9 None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:59:08.922991 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9b807a-848f-4a82-b219-4dd7488a24fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:59:08.923659 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9b807a-848f-4a82-b219-4dd7488a24fe None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:59:08.923976 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9b807a-848f-4a82-b219-4dd7488a24fe None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:59:08.924256 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9b807a-848f-4a82-b219-4dd7488a24fe None admin] PATH_INFO: `/v3/domains` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:59:08.924985 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9b807a-848f-4a82-b219-4dd7488a24fe None admin] RBAC: Adding query filter params (name=Default) {{(pid=80539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 11:59:08.925292 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9b807a-848f-4a82-b219-4dd7488a24fe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:59:08.927099 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9b807a-848f-4a82-b219-4dd7488a24fe None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:59:08.947200 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 510/1034] 10.222.0.23 () {56 vars in 1109 bytes} [Thu May 19 11:59:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:59:08.948073 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9b807a-848f-4a82-b219-4dd7488a24fe None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:59:09.033776 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e712537-8d23-41c0-bc95-6829d706d0f9 None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:59:09.055406 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:59:09.062669 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9b807a-848f-4a82-b219-4dd7488a24fe None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:59:09.071907 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a823c2d-edbc-4ce0-9b2c-f05f586fbca4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:59:09.071907 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a823c2d-edbc-4ce0-9b2c-f05f586fbca4 None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:59:09.071907 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a823c2d-edbc-4ce0-9b2c-f05f586fbca4 None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:59:09.071907 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a823c2d-edbc-4ce0-9b2c-f05f586fbca4 None admin] PATH_INFO: `/v3/services` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:59:09.072257 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a823c2d-edbc-4ce0-9b2c-f05f586fbca4 None admin] RBAC: Adding query filter params (type=image) {{(pid=80540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 11:59:09.072257 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a823c2d-edbc-4ce0-9b2c-f05f586fbca4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:59:09.073480 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a823c2d-edbc-4ce0-9b2c-f05f586fbca4 None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:59:09.079891 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 525/1035] 10.222.0.23 () {56 vars in 1108 bytes} [Thu May 19 11:59:08 2022] GET /identity/v3/services?type=image => generated 350 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 11:59:09.085666 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a823c2d-edbc-4ce0-9b2c-f05f586fbca4 None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:59:09.099747 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be58c34-b538-46e7-94d6-74ba4fa973ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:59:09.100590 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be58c34-b538-46e7-94d6-74ba4fa973ea None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:59:09.100875 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be58c34-b538-46e7-94d6-74ba4fa973ea None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:59:09.101151 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be58c34-b538-46e7-94d6-74ba4fa973ea None admin] PATH_INFO: `/v3/users/nova` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:59:09.109886 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be58c34-b538-46e7-94d6-74ba4fa973ea None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:59:09.111334 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be58c34-b538-46e7-94d6-74ba4fa973ea None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:59:09.126583 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application [None req-6be58c34-b538-46e7-94d6-74ba4fa973ea None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 19 11:59:09.126583 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 11:59:09.126583 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 11:59:09.126583 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 11:59:09.126583 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 11:59:09.126583 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 11:59:09.126583 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 11:59:09.126583 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 11:59:09.126583 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 11:59:09.126583 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 11:59:09.126583 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 11:59:09.126583 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 11:59:09.126583 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 11:59:09.126583 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 11:59:09.126583 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 11:59:09.126583 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 11:59:09.127850 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 11:59:09.127850 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 11:59:09.127850 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 11:59:09.127850 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 11:59:09.127850 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 11:59:09.127850 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 11:59:09.127850 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 11:59:09.127850 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 11:59:09.127850 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 11:59:09.127850 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.get_or_create( May 19 11:59:09.127850 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 11:59:09.127850 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application with Lock( May 19 11:59:09.127850 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 11:59:09.127850 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._enter() May 19 11:59:09.127850 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 11:59:09.127850 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 11:59:09.127850 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 11:59:09.127850 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.creator() May 19 11:59:09.128816 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 11:59:09.128816 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application created_value = creator( May 19 11:59:09.128816 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 11:59:09.128816 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 11:59:09.128816 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 11:59:09.128816 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 11:59:09.128816 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 11:59:09.128816 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 11:59:09.128816 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 19 11:59:09.128816 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application May 19 11:59:09.128816 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 511/1036] 10.222.0.23 () {56 vars in 1093 bytes} [Thu May 19 11:59:08 2022] GET /identity/v3/users/nova => generated 82 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 11:59:09.133540 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be58c34-b538-46e7-94d6-74ba4fa973ea None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:59:09.158587 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:59:09.202118 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a823c2d-edbc-4ce0-9b2c-f05f586fbca4 None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:59:09.239639 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c50918-d8fe-4939-8da9-4ad68fcc2133 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:59:09.240649 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c50918-d8fe-4939-8da9-4ad68fcc2133 None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:59:09.241113 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c50918-d8fe-4939-8da9-4ad68fcc2133 None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:59:09.241394 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c50918-d8fe-4939-8da9-4ad68fcc2133 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:59:09.242191 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c50918-d8fe-4939-8da9-4ad68fcc2133 None admin] RBAC: Adding query filter params (interface=public, service_id=6768fa4c75d54f7d9b1c0de563699997, region_id=RegionOne) {{(pid=80540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 11:59:09.242510 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c50918-d8fe-4939-8da9-4ad68fcc2133 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:59:09.243461 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be58c34-b538-46e7-94d6-74ba4fa973ea None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:59:09.243928 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c50918-d8fe-4939-8da9-4ad68fcc2133 None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:59:09.254926 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 526/1037] 10.222.0.23 () {56 vars in 1251 bytes} [Thu May 19 11:59:09 2022] GET /identity/v3/endpoints?service_id=6768fa4c75d54f7d9b1c0de563699997&interface=public®ion_id=RegionOne => generated 194 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:59:09.255975 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c50918-d8fe-4939-8da9-4ad68fcc2133 None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:59:09.285702 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7faa285-cda0-4fa7-bce6-c7f7f2d40084 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:59:09.286378 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7faa285-cda0-4fa7-bce6-c7f7f2d40084 None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:59:09.286667 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7faa285-cda0-4fa7-bce6-c7f7f2d40084 None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:59:09.286952 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7faa285-cda0-4fa7-bce6-c7f7f2d40084 None admin] PATH_INFO: `/v3/users` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:59:09.287813 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7faa285-cda0-4fa7-bce6-c7f7f2d40084 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 11:59:09.288140 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7faa285-cda0-4fa7-bce6-c7f7f2d40084 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:59:09.289568 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7faa285-cda0-4fa7-bce6-c7f7f2d40084 None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:59:09.307572 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 512/1038] 10.222.0.23 () {56 vars in 1133 bytes} [Thu May 19 11:59:09 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 11:59:09.313864 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7faa285-cda0-4fa7-bce6-c7f7f2d40084 None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:59:09.329718 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:59:09.366604 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c50918-d8fe-4939-8da9-4ad68fcc2133 None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:59:09.394095 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:59:09.403599 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e769e9f-4d47-411c-9f4b-fd1dc4b54a21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:59:09.404425 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e769e9f-4d47-411c-9f4b-fd1dc4b54a21 None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:59:09.404605 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e769e9f-4d47-411c-9f4b-fd1dc4b54a21 None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:59:09.405996 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e769e9f-4d47-411c-9f4b-fd1dc4b54a21 None admin] PATH_INFO: `/v3/services` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:59:09.405996 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e769e9f-4d47-411c-9f4b-fd1dc4b54a21 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:59:09.407029 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e769e9f-4d47-411c-9f4b-fd1dc4b54a21 None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:59:09.413923 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 527/1039] 10.222.0.23 () {56 vars in 1087 bytes} [Thu May 19 11:59:09 2022] GET /identity/v3/services => generated 1957 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 19 11:59:09.432574 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7faa285-cda0-4fa7-bce6-c7f7f2d40084 None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:59:09.474032 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb99473-15a1-4b18-96eb-4a1232d56038 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:59:09.474624 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb99473-15a1-4b18-96eb-4a1232d56038 None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:59:09.474917 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb99473-15a1-4b18-96eb-4a1232d56038 None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:59:09.475198 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb99473-15a1-4b18-96eb-4a1232d56038 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:59:09.557191 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb99473-15a1-4b18-96eb-4a1232d56038 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:59:09.558771 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb99473-15a1-4b18-96eb-4a1232d56038 None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:59:09.578004 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:59:09.673993 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 513/1040] 10.222.0.23 () {58 vars in 1282 bytes} [Thu May 19 11:59:09 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 19 11:59:09.679699 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e769e9f-4d47-411c-9f4b-fd1dc4b54a21 None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:59:09.784557 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e769e9f-4d47-411c-9f4b-fd1dc4b54a21 None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:59:09.819014 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076e4a04-6b04-4b30-bbfd-9c37a2bdbd5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:59:09.819831 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076e4a04-6b04-4b30-bbfd-9c37a2bdbd5c None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:59:09.820125 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076e4a04-6b04-4b30-bbfd-9c37a2bdbd5c None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:59:09.820405 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076e4a04-6b04-4b30-bbfd-9c37a2bdbd5c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:59:09.832142 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-076e4a04-6b04-4b30-bbfd-9c37a2bdbd5c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:59:09.833621 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-076e4a04-6b04-4b30-bbfd-9c37a2bdbd5c None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:59:09.850982 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application [None req-076e4a04-6b04-4b30-bbfd-9c37a2bdbd5c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 11:59:09.850982 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 11:59:09.850982 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 11:59:09.850982 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 11:59:09.850982 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 11:59:09.850982 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 11:59:09.850982 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 11:59:09.850982 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 11:59:09.850982 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 11:59:09.850982 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 11:59:09.850982 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 11:59:09.850982 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 11:59:09.850982 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 11:59:09.850982 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 11:59:09.850982 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 11:59:09.851967 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 11:59:09.851967 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 11:59:09.851967 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 11:59:09.851967 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 11:59:09.851967 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 11:59:09.851967 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 11:59:09.851967 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.get_or_create( May 19 11:59:09.851967 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 11:59:09.851967 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application with Lock( May 19 11:59:09.851967 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 11:59:09.851967 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._enter() May 19 11:59:09.851967 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 11:59:09.851967 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 11:59:09.851967 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 11:59:09.851967 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.creator() May 19 11:59:09.851967 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 11:59:09.851967 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application created_value = creator( May 19 11:59:09.852925 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 11:59:09.852925 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 11:59:09.852925 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 11:59:09.852925 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application May 19 11:59:09.852925 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 528/1041] 10.222.0.23 () {56 vars in 1108 bytes} [Thu May 19 11:59:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 11:59:09.859417 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb99473-15a1-4b18-96eb-4a1232d56038 None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:59:09.975937 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb99473-15a1-4b18-96eb-4a1232d56038 None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:59:09.992474 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:59:10.015908 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09647e1-da6e-41aa-8bce-c26fd733343a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:59:10.016595 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09647e1-da6e-41aa-8bce-c26fd733343a None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:59:10.016881 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09647e1-da6e-41aa-8bce-c26fd733343a None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:59:10.017190 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09647e1-da6e-41aa-8bce-c26fd733343a None admin] PATH_INFO: `/v3/domains` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:59:10.017949 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09647e1-da6e-41aa-8bce-c26fd733343a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 11:59:10.018260 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09647e1-da6e-41aa-8bce-c26fd733343a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:59:10.019658 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09647e1-da6e-41aa-8bce-c26fd733343a None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:59:10.037597 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 514/1042] 10.222.0.23 () {56 vars in 1109 bytes} [Thu May 19 11:59:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:59:10.039387 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076e4a04-6b04-4b30-bbfd-9c37a2bdbd5c None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:59:10.093690 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:59:10.166752 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076e4a04-6b04-4b30-bbfd-9c37a2bdbd5c None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:59:10.203700 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54cb2a2-12cc-436c-a551-edd0be4dc331 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:59:10.204549 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54cb2a2-12cc-436c-a551-edd0be4dc331 None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:59:10.204826 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54cb2a2-12cc-436c-a551-edd0be4dc331 None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:59:10.205114 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54cb2a2-12cc-436c-a551-edd0be4dc331 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:59:10.212433 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54cb2a2-12cc-436c-a551-edd0be4dc331 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:59:10.213982 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54cb2a2-12cc-436c-a551-edd0be4dc331 None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:59:10.229794 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application [None req-d54cb2a2-12cc-436c-a551-edd0be4dc331 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 11:59:10.229794 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 11:59:10.229794 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 11:59:10.229794 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 11:59:10.229794 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 11:59:10.229794 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 11:59:10.229794 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 11:59:10.229794 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 11:59:10.229794 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 11:59:10.229794 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 11:59:10.229794 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 11:59:10.229794 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 11:59:10.229794 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 11:59:10.229794 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 11:59:10.229794 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 11:59:10.230768 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 11:59:10.230768 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 11:59:10.230768 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 11:59:10.230768 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 11:59:10.230768 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 11:59:10.230768 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 11:59:10.230768 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.get_or_create( May 19 11:59:10.230768 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 11:59:10.230768 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application with Lock( May 19 11:59:10.230768 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 11:59:10.230768 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._enter() May 19 11:59:10.230768 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 11:59:10.230768 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 11:59:10.230768 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 11:59:10.230768 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.creator() May 19 11:59:10.230768 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 11:59:10.230768 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application created_value = creator( May 19 11:59:10.231730 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 11:59:10.231730 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 11:59:10.231730 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 11:59:10.231730 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 11:59:10.231730 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 11:59:10.231730 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 11:59:10.231730 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 11:59:10.231730 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application May 19 11:59:10.236247 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 529/1043] 10.222.0.23 () {56 vars in 1111 bytes} [Thu May 19 11:59:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 11:59:10.237406 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09647e1-da6e-41aa-8bce-c26fd733343a None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:59:10.344302 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09647e1-da6e-41aa-8bce-c26fd733343a None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:59:10.383508 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3be27e7-89c9-4feb-bad9-b4a954aaca97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:59:10.384234 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3be27e7-89c9-4feb-bad9-b4a954aaca97 None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:59:10.384531 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3be27e7-89c9-4feb-bad9-b4a954aaca97 None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:59:10.384816 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3be27e7-89c9-4feb-bad9-b4a954aaca97 None admin] PATH_INFO: `/v3/projects` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:59:10.385637 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3be27e7-89c9-4feb-bad9-b4a954aaca97 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 11:59:10.385953 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3be27e7-89c9-4feb-bad9-b4a954aaca97 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:59:10.387448 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3be27e7-89c9-4feb-bad9-b4a954aaca97 None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:59:10.407657 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 515/1044] 10.222.0.23 () {56 vars in 1148 bytes} [Thu May 19 11:59:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 11:59:10.409285 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54cb2a2-12cc-436c-a551-edd0be4dc331 None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:59:10.514471 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54cb2a2-12cc-436c-a551-edd0be4dc331 None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:59:10.516098 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:59:10.550898 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf3143f-1428-4898-b15f-fdd0fa424125 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:59:10.551558 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf3143f-1428-4898-b15f-fdd0fa424125 None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:59:10.551916 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf3143f-1428-4898-b15f-fdd0fa424125 None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:59:10.552160 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf3143f-1428-4898-b15f-fdd0fa424125 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:59:10.553081 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf3143f-1428-4898-b15f-fdd0fa424125 None admin] RBAC: Adding query filter params (role.id=ff2dfc5f80304d9cbd3320d8ceffe2a2, scope.project.id=d70d25bf69314820b8aaecb65576978a, user.id=34a98d75d84b4c67981377e0f26eca62) {{(pid=80540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 11:59:10.553410 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf3143f-1428-4898-b15f-fdd0fa424125 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:59:10.554905 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf3143f-1428-4898-b15f-fdd0fa424125 None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:59:10.567835 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 530/1045] 10.222.0.23 () {56 vars in 1374 bytes} [Thu May 19 11:59:10 2022] GET /identity/v3/role_assignments?user.id=34a98d75d84b4c67981377e0f26eca62&scope.project.id=d70d25bf69314820b8aaecb65576978a&role.id=ff2dfc5f80304d9cbd3320d8ceffe2a2 => generated 259 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:59:10.576047 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac43c0d-e08d-4a85-9710-7af78427084d None None] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:59:10.576335 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac43c0d-e08d-4a85-9710-7af78427084d None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:59:10.576605 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac43c0d-e08d-4a85-9710-7af78427084d None None] PATH_INFO: `/` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:59:10.578248 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 516/1046] 10.222.0.23 () {54 vars in 926 bytes} [Thu May 19 11:59:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 11:59:10.584270 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a42d04-c51d-4f0d-a31e-a10f016359f1 None None] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:59:10.584606 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a42d04-c51d-4f0d-a31e-a10f016359f1 None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:59:10.584848 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a42d04-c51d-4f0d-a31e-a10f016359f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:59:10.603416 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.auth.core [None req-00a42d04-c51d-4f0d-a31e-a10f016359f1 None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 11:59:10.604583 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-00a42d04-c51d-4f0d-a31e-a10f016359f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 11:59:10.683754 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 531/1047] 10.222.0.23 () {58 vars in 1023 bytes} [Thu May 19 11:59:10 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 11:59:10.708500 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac43c0d-e08d-4a85-9710-7af78427084d None None] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:59:10.748504 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:59:10.818447 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac43c0d-e08d-4a85-9710-7af78427084d None None] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:59:10.854869 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563646de-3704-4545-9dc3-2bd54fb34d14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:59:10.855578 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563646de-3704-4545-9dc3-2bd54fb34d14 None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:59:10.855889 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563646de-3704-4545-9dc3-2bd54fb34d14 None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:59:10.856169 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563646de-3704-4545-9dc3-2bd54fb34d14 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:59:10.867164 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563646de-3704-4545-9dc3-2bd54fb34d14 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 11:59:10.875072 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application [None req-563646de-3704-4545-9dc3-2bd54fb34d14 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 11:59:10.875072 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 11:59:10.875072 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 11:59:10.875072 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 11:59:10.875072 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 11:59:10.875072 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 11:59:10.875072 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 11:59:10.875072 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 11:59:10.875072 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 11:59:10.875072 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 11:59:10.875072 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 11:59:10.875072 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 11:59:10.875072 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 11:59:10.875072 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 11:59:10.875072 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 11:59:10.875072 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 11:59:10.876081 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 11:59:10.876081 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 11:59:10.876081 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 11:59:10.876081 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 11:59:10.876081 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 11:59:10.876081 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 11:59:10.876081 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 11:59:10.876081 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 11:59:10.876081 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 11:59:10.876081 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.get_or_create( May 19 11:59:10.876081 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 11:59:10.876081 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application with Lock( May 19 11:59:10.876081 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 11:59:10.876081 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._enter() May 19 11:59:10.876081 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 11:59:10.876081 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 11:59:10.876923 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 11:59:10.876923 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.creator() May 19 11:59:10.876923 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 11:59:10.876923 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application created_value = creator( May 19 11:59:10.876923 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 11:59:10.876923 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 11:59:10.876923 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 11:59:10.876923 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 11:59:10.876923 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 11:59:10.876923 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 11:59:10.876923 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 11:59:10.876923 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application May 19 11:59:10.876923 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 517/1048] 10.222.0.23 () {56 vars in 1096 bytes} [Thu May 19 11:59:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 11:59:10.883590 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a42d04-c51d-4f0d-a31e-a10f016359f1 None None] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:59:10.991572 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:59:10.998296 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a42d04-c51d-4f0d-a31e-a10f016359f1 None None] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:59:11.033563 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f03e8f2-b40e-4b04-9f07-9324723013d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:59:11.034249 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f03e8f2-b40e-4b04-9f07-9324723013d7 None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:59:11.034535 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f03e8f2-b40e-4b04-9f07-9324723013d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:59:11.034813 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f03e8f2-b40e-4b04-9f07-9324723013d7 None admin] PATH_INFO: `/v3/roles` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:59:11.035576 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f03e8f2-b40e-4b04-9f07-9324723013d7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 11:59:11.035991 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f03e8f2-b40e-4b04-9f07-9324723013d7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:59:11.037436 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f03e8f2-b40e-4b04-9f07-9324723013d7 None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:59:11.054250 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 532/1049] 10.222.0.23 () {56 vars in 1099 bytes} [Thu May 19 11:59:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 11:59:11.055750 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563646de-3704-4545-9dc3-2bd54fb34d14 None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:59:11.155567 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:59:11.162970 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563646de-3704-4545-9dc3-2bd54fb34d14 None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:59:11.197844 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594627ca-2c0c-48e4-b615-fe2df47e22be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:59:11.198728 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594627ca-2c0c-48e4-b615-fe2df47e22be None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:59:11.199007 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594627ca-2c0c-48e4-b615-fe2df47e22be None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:59:11.199279 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594627ca-2c0c-48e4-b615-fe2df47e22be None admin] PATH_INFO: `/v3/users/c1068d87fabf43b2ad94802b55feeb62` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:59:11.213183 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594627ca-2c0c-48e4-b615-fe2df47e22be None admin] RBAC: Authorizing `identity:get_user(user_id=c1068d87fabf43b2ad94802b55feeb62)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:59:11.214625 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594627ca-2c0c-48e4-b615-fe2df47e22be None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:59:11.217044 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 518/1050] 10.222.0.23 () {56 vars in 1177 bytes} [Thu May 19 11:59:11 2022] GET /identity/v3/users/c1068d87fabf43b2ad94802b55feeb62 => generated 285 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 11:59:11.222531 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f03e8f2-b40e-4b04-9f07-9324723013d7 None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:59:11.326428 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f03e8f2-b40e-4b04-9f07-9324723013d7 None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:59:11.360224 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fbd41c-3adb-418b-852a-46f95d75bfec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:59:11.360932 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fbd41c-3adb-418b-852a-46f95d75bfec None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:59:11.361169 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fbd41c-3adb-418b-852a-46f95d75bfec None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:59:11.361453 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fbd41c-3adb-418b-852a-46f95d75bfec None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:59:11.389502 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:59:11.432176 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36fbd41c-3adb-418b-852a-46f95d75bfec None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:59:11.433801 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36fbd41c-3adb-418b-852a-46f95d75bfec None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:59:11.552619 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 533/1051] 10.222.0.23 () {58 vars in 1282 bytes} [Thu May 19 11:59:11 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 11:59:11.554221 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594627ca-2c0c-48e4-b615-fe2df47e22be None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:59:11.674947 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594627ca-2c0c-48e4-b615-fe2df47e22be None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:59:11.695565 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551ef835-b471-4bac-895f-2c86a3342382 None None] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:59:11.696107 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551ef835-b471-4bac-895f-2c86a3342382 None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:59:11.696107 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551ef835-b471-4bac-895f-2c86a3342382 None None] PATH_INFO: `/` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:59:11.697867 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 534/1052] 10.222.0.23 () {54 vars in 926 bytes} [Thu May 19 11:59:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 11:59:11.704296 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9572e6a0-f124-4064-994a-5b1335b9f30e None None] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:59:11.704591 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9572e6a0-f124-4064-994a-5b1335b9f30e None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:59:11.704873 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9572e6a0-f124-4064-994a-5b1335b9f30e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:59:11.717123 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92139c6b-5c3d-4e5f-bda9-b7c736ee1ef0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:59:11.718556 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92139c6b-5c3d-4e5f-bda9-b7c736ee1ef0 None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:59:11.718844 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92139c6b-5c3d-4e5f-bda9-b7c736ee1ef0 None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:59:11.719121 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92139c6b-5c3d-4e5f-bda9-b7c736ee1ef0 None admin] PATH_INFO: `/v3/projects/545d6fe0ae914bcb891a927c746bd18f` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:59:11.720411 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92139c6b-5c3d-4e5f-bda9-b7c736ee1ef0 None admin] RBAC: Authorizing `identity:get_project(project_id=545d6fe0ae914bcb891a927c746bd18f)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:59:11.721904 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92139c6b-5c3d-4e5f-bda9-b7c736ee1ef0 None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:59:11.728825 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 519/1053] 10.222.0.23 () {56 vars in 1186 bytes} [Thu May 19 11:59:11 2022] GET /identity/v3/projects/545d6fe0ae914bcb891a927c746bd18f => generated 311 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 11:59:11.728990 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.auth.core [None req-9572e6a0-f124-4064-994a-5b1335b9f30e None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 11:59:11.728990 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-9572e6a0-f124-4064-994a-5b1335b9f30e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 11:59:11.729731 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92139c6b-5c3d-4e5f-bda9-b7c736ee1ef0 None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:59:11.760234 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:59:11.799083 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:59:11.815817 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 535/1054] 10.222.0.23 () {58 vars in 1023 bytes} [Thu May 19 11:59:11 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 11:59:11.840593 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9572e6a0-f124-4064-994a-5b1335b9f30e None None] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:59:11.856399 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92139c6b-5c3d-4e5f-bda9-b7c736ee1ef0 None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:59:11.901099 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce76e55-decd-4d92-842a-4fae8bb925d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:59:11.901868 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce76e55-decd-4d92-842a-4fae8bb925d1 None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:59:11.902225 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce76e55-decd-4d92-842a-4fae8bb925d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:59:11.902509 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce76e55-decd-4d92-842a-4fae8bb925d1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:59:11.903358 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce76e55-decd-4d92-842a-4fae8bb925d1 None admin] RBAC: Adding query filter params (role.id=9db5671c65d3414ea2682afa2ccf1403, scope.project.id=545d6fe0ae914bcb891a927c746bd18f, user.id=c1068d87fabf43b2ad94802b55feeb62) {{(pid=80539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 11:59:11.904473 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce76e55-decd-4d92-842a-4fae8bb925d1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:59:11.906096 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce76e55-decd-4d92-842a-4fae8bb925d1 None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:59:11.920764 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 520/1055] 10.222.0.23 () {56 vars in 1374 bytes} [Thu May 19 11:59:11 2022] GET /identity/v3/role_assignments?user.id=c1068d87fabf43b2ad94802b55feeb62&scope.project.id=545d6fe0ae914bcb891a927c746bd18f&role.id=9db5671c65d3414ea2682afa2ccf1403 => generated 259 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 11:59:11.985674 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9572e6a0-f124-4064-994a-5b1335b9f30e None None] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:59:12.031148 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da43f3c1-2068-4e2d-a6c0-7d9dc03b46b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:59:12.032007 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da43f3c1-2068-4e2d-a6c0-7d9dc03b46b2 None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:59:12.032294 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da43f3c1-2068-4e2d-a6c0-7d9dc03b46b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:59:12.032570 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da43f3c1-2068-4e2d-a6c0-7d9dc03b46b2 None admin] PATH_INFO: `/v3/services/image` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:59:12.033249 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da43f3c1-2068-4e2d-a6c0-7d9dc03b46b2 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:59:12.034706 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da43f3c1-2068-4e2d-a6c0-7d9dc03b46b2 None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:59:12.047990 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application [None req-da43f3c1-2068-4e2d-a6c0-7d9dc03b46b2 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. May 19 11:59:12.047990 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 11:59:12.047990 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 11:59:12.047990 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 11:59:12.047990 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 11:59:12.047990 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 11:59:12.047990 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 11:59:12.047990 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 11:59:12.047990 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 19 11:59:12.047990 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application May 19 11:59:12.047990 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 11:59:12.047990 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application May 19 11:59:12.047990 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 11:59:12.047990 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 11:59:12.047990 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 11:59:12.047990 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 11:59:12.047990 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 11:59:12.049272 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 11:59:12.049272 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 11:59:12.049272 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 11:59:12.049272 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 11:59:12.049272 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 11:59:12.049272 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 11:59:12.049272 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 11:59:12.049272 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 11:59:12.049272 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 11:59:12.049272 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 11:59:12.049272 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 11:59:12.049272 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 11:59:12.049272 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 11:59:12.049272 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 11:59:12.049272 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 11:59:12.049272 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.get_or_create( May 19 11:59:12.055744 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 11:59:12.055744 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application with Lock( May 19 11:59:12.055744 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 11:59:12.055744 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._enter() May 19 11:59:12.055744 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 11:59:12.055744 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 11:59:12.055744 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 11:59:12.055744 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.creator() May 19 11:59:12.055744 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 11:59:12.055744 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application created_value = creator( May 19 11:59:12.055744 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 11:59:12.055744 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 11:59:12.055744 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 19 11:59:12.055744 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application May 19 11:59:12.055744 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 536/1056] 10.222.0.23 () {56 vars in 1105 bytes} [Thu May 19 11:59:11 2022] GET /identity/v3/services/image => generated 86 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 11:59:12.056590 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce76e55-decd-4d92-842a-4fae8bb925d1 None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:59:12.166415 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:59:12.186664 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce76e55-decd-4d92-842a-4fae8bb925d1 None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:59:12.224133 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917ffe97-c670-46ad-b5dd-40790a31b8ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:59:12.224718 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917ffe97-c670-46ad-b5dd-40790a31b8ed None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:59:12.225001 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917ffe97-c670-46ad-b5dd-40790a31b8ed None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:59:12.225273 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917ffe97-c670-46ad-b5dd-40790a31b8ed None admin] PATH_INFO: `/v3/services` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:59:12.225986 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917ffe97-c670-46ad-b5dd-40790a31b8ed None admin] RBAC: Adding query filter params (name=image) {{(pid=80539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 11:59:12.226294 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917ffe97-c670-46ad-b5dd-40790a31b8ed None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:59:12.227660 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917ffe97-c670-46ad-b5dd-40790a31b8ed None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:59:12.233744 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 521/1057] 10.222.0.23 () {56 vars in 1108 bytes} [Thu May 19 11:59:12 2022] GET /identity/v3/services?name=image => generated 122 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:59:12.239201 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da43f3c1-2068-4e2d-a6c0-7d9dc03b46b2 None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:59:12.302495 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:59:12.359582 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da43f3c1-2068-4e2d-a6c0-7d9dc03b46b2 None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:59:12.397046 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac9534f-adcd-4419-8776-191bc56037ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:59:12.397711 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac9534f-adcd-4419-8776-191bc56037ab None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:59:12.397994 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac9534f-adcd-4419-8776-191bc56037ab None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:59:12.398270 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac9534f-adcd-4419-8776-191bc56037ab None admin] PATH_INFO: `/v3/services` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:59:12.399083 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac9534f-adcd-4419-8776-191bc56037ab None admin] RBAC: Adding query filter params (type=image) {{(pid=80540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 11:59:12.399384 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac9534f-adcd-4419-8776-191bc56037ab None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:59:12.400814 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac9534f-adcd-4419-8776-191bc56037ab None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:59:12.406391 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 537/1058] 10.222.0.23 () {56 vars in 1108 bytes} [Thu May 19 11:59:12 2022] GET /identity/v3/services?type=image => generated 350 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:59:12.411840 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917ffe97-c670-46ad-b5dd-40790a31b8ed None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:59:12.528706 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917ffe97-c670-46ad-b5dd-40790a31b8ed None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:59:12.570242 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165ea383-2a33-46f0-aa51-e589a3540c5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:59:12.570900 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165ea383-2a33-46f0-aa51-e589a3540c5c None admin] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:59:12.571180 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165ea383-2a33-46f0-aa51-e589a3540c5c None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:59:12.571707 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165ea383-2a33-46f0-aa51-e589a3540c5c None admin] PATH_INFO: `/v3/endpoints` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:59:12.572555 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165ea383-2a33-46f0-aa51-e589a3540c5c None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:59:12.573917 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165ea383-2a33-46f0-aa51-e589a3540c5c None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:59:12.658504 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 522/1059] 10.222.0.23 () {60 vars in 1144 bytes} [Thu May 19 11:59:12 2022] POST /identity/v3/endpoints => generated 327 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 11:59:12.850403 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee13901b-eb2a-45b0-b176-e8b7d39e66a9 None None] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:59:12.850403 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee13901b-eb2a-45b0-b176-e8b7d39e66a9 None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:59:12.851016 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee13901b-eb2a-45b0-b176-e8b7d39e66a9 None None] PATH_INFO: `/` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:59:12.852230 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 538/1060] 10.222.0.23 () {54 vars in 926 bytes} [Thu May 19 11:59:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 11:59:12.857902 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95825906-08b8-4e01-8a28-b09ad9d94b67 None None] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:59:12.858200 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95825906-08b8-4e01-8a28-b09ad9d94b67 None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:59:12.858480 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95825906-08b8-4e01-8a28-b09ad9d94b67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:59:12.864955 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:59:12.879136 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.auth.core [None req-95825906-08b8-4e01-8a28-b09ad9d94b67 None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 11:59:12.880583 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-95825906-08b8-4e01-8a28-b09ad9d94b67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 11:59:12.979825 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 523/1061] 10.222.0.23 () {58 vars in 1023 bytes} [Thu May 19 11:59:12 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 11:59:13.000296 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee13901b-eb2a-45b0-b176-e8b7d39e66a9 None None] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:59:13.126070 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee13901b-eb2a-45b0-b176-e8b7d39e66a9 None None] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:59:13.151451 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:59:13.173679 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866e1b68-4688-48f6-bee9-4ba9e66a7b5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:59:13.174582 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866e1b68-4688-48f6-bee9-4ba9e66a7b5d None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:59:13.175018 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866e1b68-4688-48f6-bee9-4ba9e66a7b5d None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:59:13.175283 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866e1b68-4688-48f6-bee9-4ba9e66a7b5d None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:59:13.188552 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866e1b68-4688-48f6-bee9-4ba9e66a7b5d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 11:59:13.197106 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application [None req-866e1b68-4688-48f6-bee9-4ba9e66a7b5d None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 19 11:59:13.197106 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 11:59:13.197106 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 11:59:13.197106 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 11:59:13.197106 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 11:59:13.197106 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 11:59:13.197106 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 11:59:13.197106 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 11:59:13.197106 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 11:59:13.197106 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 11:59:13.197106 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 11:59:13.197106 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 11:59:13.197106 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 11:59:13.197106 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 11:59:13.197106 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 11:59:13.197106 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 11:59:13.198099 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 11:59:13.198099 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 11:59:13.198099 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 11:59:13.198099 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 11:59:13.198099 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 11:59:13.198099 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 11:59:13.198099 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 11:59:13.198099 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 11:59:13.198099 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 11:59:13.198099 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.get_or_create( May 19 11:59:13.198099 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 11:59:13.198099 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application with Lock( May 19 11:59:13.198099 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 11:59:13.198099 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._enter() May 19 11:59:13.198099 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 11:59:13.198099 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 11:59:13.198940 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 11:59:13.198940 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.creator() May 19 11:59:13.198940 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 11:59:13.198940 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application created_value = creator( May 19 11:59:13.198940 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 11:59:13.198940 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 11:59:13.198940 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 11:59:13.198940 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 11:59:13.198940 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 11:59:13.198940 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 11:59:13.198940 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 19 11:59:13.198940 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application May 19 11:59:13.203523 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 539/1062] 10.222.0.23 () {56 vars in 1120 bytes} [Thu May 19 11:59:12 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 11:59:13.204596 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95825906-08b8-4e01-8a28-b09ad9d94b67 None None] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:59:13.332068 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95825906-08b8-4e01-8a28-b09ad9d94b67 None None] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:59:13.368381 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:59:13.383940 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4c1d63-3bd5-4dcf-88d7-c0436c2c637b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:59:13.383940 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4c1d63-3bd5-4dcf-88d7-c0436c2c637b None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:59:13.383940 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4c1d63-3bd5-4dcf-88d7-c0436c2c637b None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:59:13.383940 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4c1d63-3bd5-4dcf-88d7-c0436c2c637b None admin] PATH_INFO: `/v3/roles` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:59:13.384333 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4c1d63-3bd5-4dcf-88d7-c0436c2c637b None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 11:59:13.384333 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4c1d63-3bd5-4dcf-88d7-c0436c2c637b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:59:13.384333 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4c1d63-3bd5-4dcf-88d7-c0436c2c637b None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:59:13.389861 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 524/1063] 10.222.0.23 () {56 vars in 1115 bytes} [Thu May 19 11:59:13 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:59:13.395646 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866e1b68-4688-48f6-bee9-4ba9e66a7b5d None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:59:13.515594 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866e1b68-4688-48f6-bee9-4ba9e66a7b5d None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:59:13.560503 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab923d8-5900-4c81-8c15-349ce1950f56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:59:13.561559 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab923d8-5900-4c81-8c15-349ce1950f56 None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:59:13.561846 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab923d8-5900-4c81-8c15-349ce1950f56 None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:59:13.562124 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab923d8-5900-4c81-8c15-349ce1950f56 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:59:13.576239 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab923d8-5900-4c81-8c15-349ce1950f56 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:59:13.577731 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab923d8-5900-4c81-8c15-349ce1950f56 None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:59:13.597493 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application [None req-7ab923d8-5900-4c81-8c15-349ce1950f56 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 11:59:13.597493 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 11:59:13.597493 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 11:59:13.597493 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 11:59:13.597493 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 11:59:13.597493 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 11:59:13.597493 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 11:59:13.597493 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 11:59:13.597493 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 11:59:13.597493 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 11:59:13.597493 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 11:59:13.597493 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 11:59:13.597493 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 11:59:13.597493 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 11:59:13.597493 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 11:59:13.598490 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 11:59:13.598490 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 11:59:13.598490 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 11:59:13.598490 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 11:59:13.598490 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 11:59:13.598490 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 11:59:13.598490 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.get_or_create( May 19 11:59:13.598490 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 11:59:13.598490 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application with Lock( May 19 11:59:13.598490 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 11:59:13.598490 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._enter() May 19 11:59:13.598490 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 11:59:13.598490 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 11:59:13.598490 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 11:59:13.598490 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.creator() May 19 11:59:13.598490 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 11:59:13.598490 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application created_value = creator( May 19 11:59:13.599400 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 11:59:13.599400 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 11:59:13.599400 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 11:59:13.599400 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application May 19 11:59:13.603451 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 540/1064] 10.222.0.23 () {56 vars in 1108 bytes} [Thu May 19 11:59:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 11:59:13.604867 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4c1d63-3bd5-4dcf-88d7-c0436c2c637b None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:59:13.724615 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4c1d63-3bd5-4dcf-88d7-c0436c2c637b None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:59:13.762563 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52cdf8ce-a91c-4dc6-8d2a-4c440d346101 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:59:13.763823 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cdf8ce-a91c-4dc6-8d2a-4c440d346101 None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:59:13.764112 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cdf8ce-a91c-4dc6-8d2a-4c440d346101 None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:59:13.764394 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cdf8ce-a91c-4dc6-8d2a-4c440d346101 None admin] PATH_INFO: `/v3/domains` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:59:13.765128 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52cdf8ce-a91c-4dc6-8d2a-4c440d346101 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 11:59:13.765438 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52cdf8ce-a91c-4dc6-8d2a-4c440d346101 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:59:13.766831 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52cdf8ce-a91c-4dc6-8d2a-4c440d346101 None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:59:13.781101 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 525/1065] 10.222.0.23 () {56 vars in 1109 bytes} [Thu May 19 11:59:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 11:59:13.787739 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab923d8-5900-4c81-8c15-349ce1950f56 None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:59:13.833982 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:59:13.915630 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab923d8-5900-4c81-8c15-349ce1950f56 None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:59:13.963107 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2094213d-7eea-49e8-8f96-9ff005901e8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:59:13.963940 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2094213d-7eea-49e8-8f96-9ff005901e8b None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:59:13.964223 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2094213d-7eea-49e8-8f96-9ff005901e8b None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:59:13.964496 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2094213d-7eea-49e8-8f96-9ff005901e8b None admin] PATH_INFO: `/v3/users/nova` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:59:13.973878 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2094213d-7eea-49e8-8f96-9ff005901e8b None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:59:13.975600 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2094213d-7eea-49e8-8f96-9ff005901e8b None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:59:13.994014 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application [None req-2094213d-7eea-49e8-8f96-9ff005901e8b None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 19 11:59:13.994014 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 11:59:13.994014 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 11:59:13.994014 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 11:59:13.994014 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 11:59:13.994014 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 11:59:13.994014 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 11:59:13.994014 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 11:59:13.994014 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 11:59:13.994014 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 11:59:13.994014 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 11:59:13.994014 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 11:59:13.994014 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 11:59:13.994014 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 11:59:13.994014 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 11:59:13.994014 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 11:59:13.995059 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 11:59:13.995059 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 11:59:13.995059 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 11:59:13.995059 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 11:59:13.995059 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 11:59:13.995059 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 11:59:13.995059 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 11:59:13.995059 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 11:59:13.995059 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 11:59:13.995059 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.get_or_create( May 19 11:59:13.995059 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 11:59:13.995059 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application with Lock( May 19 11:59:13.995059 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 11:59:13.995059 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._enter() May 19 11:59:13.995059 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 11:59:13.995059 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 11:59:13.995059 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 11:59:13.995059 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.creator() May 19 11:59:13.996662 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 11:59:13.996662 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application created_value = creator( May 19 11:59:13.996662 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 11:59:13.996662 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 11:59:13.996662 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 11:59:13.996662 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 11:59:13.996662 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 11:59:13.996662 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 11:59:13.996662 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 19 11:59:13.996662 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application May 19 11:59:13.996662 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 541/1066] 10.222.0.23 () {56 vars in 1093 bytes} [Thu May 19 11:59:13 2022] GET /identity/v3/users/nova => generated 82 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 11:59:14.001662 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52cdf8ce-a91c-4dc6-8d2a-4c440d346101 None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:59:14.066597 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:59:14.130197 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52cdf8ce-a91c-4dc6-8d2a-4c440d346101 None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:59:14.173290 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880cad0a-b73c-4e55-b854-76ee928200fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:59:14.173969 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880cad0a-b73c-4e55-b854-76ee928200fd None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:59:14.174250 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880cad0a-b73c-4e55-b854-76ee928200fd None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:59:14.174524 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880cad0a-b73c-4e55-b854-76ee928200fd None admin] PATH_INFO: `/v3/users` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:59:14.175365 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880cad0a-b73c-4e55-b854-76ee928200fd None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 11:59:14.175692 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880cad0a-b73c-4e55-b854-76ee928200fd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:59:14.177164 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880cad0a-b73c-4e55-b854-76ee928200fd None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:59:14.202885 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 526/1067] 10.222.0.23 () {56 vars in 1133 bytes} [Thu May 19 11:59:14 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 11:59:14.204732 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2094213d-7eea-49e8-8f96-9ff005901e8b None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:59:14.282481 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b498b92-b326-49db-a76a-b22d89b3175f None None] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:59:14.282668 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b498b92-b326-49db-a76a-b22d89b3175f None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:59:14.282952 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b498b92-b326-49db-a76a-b22d89b3175f None None] PATH_INFO: `/` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:59:14.284535 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 527/1068] 10.222.0.23 () {54 vars in 926 bytes} [Thu May 19 11:59:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 11:59:14.289906 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c15f15e-342a-48f6-b7d5-1a554a89c6a5 None None] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:59:14.290204 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c15f15e-342a-48f6-b7d5-1a554a89c6a5 None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:59:14.290495 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c15f15e-342a-48f6-b7d5-1a554a89c6a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:59:14.308227 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.auth.core [None req-2c15f15e-342a-48f6-b7d5-1a554a89c6a5 None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 11:59:14.309476 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-2c15f15e-342a-48f6-b7d5-1a554a89c6a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 11:59:14.311165 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:59:14.329406 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2094213d-7eea-49e8-8f96-9ff005901e8b None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:59:14.369722 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c57c34-029d-424f-bf51-72f7c9ebf8c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:59:14.370518 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c57c34-029d-424f-bf51-72f7c9ebf8c1 None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:59:14.370800 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c57c34-029d-424f-bf51-72f7c9ebf8c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:59:14.371072 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c57c34-029d-424f-bf51-72f7c9ebf8c1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:59:14.382132 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c57c34-029d-424f-bf51-72f7c9ebf8c1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:59:14.383606 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c57c34-029d-424f-bf51-72f7c9ebf8c1 None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:59:14.390095 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 528/1069] 10.222.0.23 () {58 vars in 1023 bytes} [Thu May 19 11:59:14 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 11:59:14.405413 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application [None req-25c57c34-029d-424f-bf51-72f7c9ebf8c1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 11:59:14.405413 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 11:59:14.405413 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 11:59:14.405413 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 11:59:14.405413 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 11:59:14.405413 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 11:59:14.405413 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 11:59:14.405413 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 11:59:14.405413 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 11:59:14.405413 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 11:59:14.405413 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 11:59:14.405413 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 11:59:14.405413 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 11:59:14.405413 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 11:59:14.405413 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 11:59:14.406371 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 11:59:14.406371 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 11:59:14.406371 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 11:59:14.406371 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 11:59:14.406371 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 11:59:14.406371 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 11:59:14.406371 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.get_or_create( May 19 11:59:14.406371 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 11:59:14.406371 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application with Lock( May 19 11:59:14.406371 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 11:59:14.406371 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._enter() May 19 11:59:14.406371 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 11:59:14.406371 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 11:59:14.406371 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 11:59:14.406371 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.creator() May 19 11:59:14.406371 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 11:59:14.406371 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application created_value = creator( May 19 11:59:14.407256 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 11:59:14.407256 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 11:59:14.407256 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 11:59:14.407256 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application May 19 11:59:14.411334 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 542/1070] 10.222.0.23 () {56 vars in 1108 bytes} [Thu May 19 11:59:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 11:59:14.412618 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c57c34-029d-424f-bf51-72f7c9ebf8c1 None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:59:14.412853 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c15f15e-342a-48f6-b7d5-1a554a89c6a5 None None] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:59:14.462227 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:59:14.535908 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c15f15e-342a-48f6-b7d5-1a554a89c6a5 None None] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:59:14.540812 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c57c34-029d-424f-bf51-72f7c9ebf8c1 None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:59:14.578337 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de49cdf-15a8-4c18-90e9-da82504c5fa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:59:14.579018 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de49cdf-15a8-4c18-90e9-da82504c5fa5 None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:59:14.579307 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de49cdf-15a8-4c18-90e9-da82504c5fa5 None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:59:14.579589 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de49cdf-15a8-4c18-90e9-da82504c5fa5 None admin] PATH_INFO: `/v3/domains` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:59:14.580368 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de49cdf-15a8-4c18-90e9-da82504c5fa5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 11:59:14.580682 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de49cdf-15a8-4c18-90e9-da82504c5fa5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:59:14.582073 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de49cdf-15a8-4c18-90e9-da82504c5fa5 None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:59:14.586712 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22da20c-fb98-4e5d-bd23-599a66a5dbc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:59:14.587515 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22da20c-fb98-4e5d-bd23-599a66a5dbc6 None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:59:14.587816 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22da20c-fb98-4e5d-bd23-599a66a5dbc6 None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:59:14.588097 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22da20c-fb98-4e5d-bd23-599a66a5dbc6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:59:14.596360 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 529/1071] 10.222.0.23 () {56 vars in 1109 bytes} [Thu May 19 11:59:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:59:14.599256 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:59:14.601727 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de49cdf-15a8-4c18-90e9-da82504c5fa5 None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:59:14.603601 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22da20c-fb98-4e5d-bd23-599a66a5dbc6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 11:59:14.611575 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application [None req-a22da20c-fb98-4e5d-bd23-599a66a5dbc6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 11:59:14.611575 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 11:59:14.611575 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 11:59:14.611575 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 11:59:14.611575 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 11:59:14.611575 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 11:59:14.611575 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 11:59:14.611575 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 11:59:14.611575 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 11:59:14.611575 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 11:59:14.611575 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 11:59:14.611575 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 11:59:14.611575 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 11:59:14.611575 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 11:59:14.611575 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 11:59:14.611575 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 11:59:14.618079 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 11:59:14.618079 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 11:59:14.618079 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 11:59:14.618079 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 11:59:14.618079 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 11:59:14.618079 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 11:59:14.618079 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 11:59:14.618079 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 11:59:14.618079 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 11:59:14.618079 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.get_or_create( May 19 11:59:14.618079 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 11:59:14.618079 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application with Lock( May 19 11:59:14.618079 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 11:59:14.618079 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._enter() May 19 11:59:14.618079 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 11:59:14.618079 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 11:59:14.619201 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 11:59:14.619201 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.creator() May 19 11:59:14.619201 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 11:59:14.619201 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application created_value = creator( May 19 11:59:14.619201 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 11:59:14.619201 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 11:59:14.619201 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 11:59:14.619201 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 11:59:14.619201 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 11:59:14.619201 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 11:59:14.619201 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 11:59:14.619201 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application May 19 11:59:14.619201 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 543/1072] 10.222.0.23 () {56 vars in 1096 bytes} [Thu May 19 11:59:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 11:59:14.620419 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22da20c-fb98-4e5d-bd23-599a66a5dbc6 None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:59:14.734971 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:59:14.738414 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de49cdf-15a8-4c18-90e9-da82504c5fa5 None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:59:14.748113 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22da20c-fb98-4e5d-bd23-599a66a5dbc6 None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:59:14.784552 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f99269-4435-4002-ad2c-de348f195594 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:59:14.785376 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f99269-4435-4002-ad2c-de348f195594 None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:59:14.785664 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f99269-4435-4002-ad2c-de348f195594 None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:59:14.786016 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f99269-4435-4002-ad2c-de348f195594 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:59:14.787971 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c17468f-b753-4029-aee3-25d9c66afdd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:59:14.788639 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c17468f-b753-4029-aee3-25d9c66afdd2 None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:59:14.788923 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c17468f-b753-4029-aee3-25d9c66afdd2 None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:59:14.789199 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c17468f-b753-4029-aee3-25d9c66afdd2 None admin] PATH_INFO: `/v3/roles` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:59:14.789950 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c17468f-b753-4029-aee3-25d9c66afdd2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 11:59:14.790253 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c17468f-b753-4029-aee3-25d9c66afdd2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:59:14.791635 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c17468f-b753-4029-aee3-25d9c66afdd2 None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:59:14.793850 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f99269-4435-4002-ad2c-de348f195594 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:59:14.795378 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f99269-4435-4002-ad2c-de348f195594 None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:59:14.801782 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 544/1073] 10.222.0.23 () {56 vars in 1099 bytes} [Thu May 19 11:59:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:59:14.807537 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c17468f-b753-4029-aee3-25d9c66afdd2 None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:59:14.811105 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application [None req-24f99269-4435-4002-ad2c-de348f195594 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 11:59:14.811105 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 11:59:14.811105 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 11:59:14.811105 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 11:59:14.811105 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 11:59:14.811105 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 11:59:14.811105 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 11:59:14.811105 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 11:59:14.811105 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 11:59:14.811105 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 11:59:14.811105 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 11:59:14.811105 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 11:59:14.811105 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 11:59:14.811105 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 11:59:14.811105 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 11:59:14.812330 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 11:59:14.812330 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 11:59:14.812330 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 11:59:14.812330 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 11:59:14.812330 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 11:59:14.812330 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 11:59:14.812330 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.get_or_create( May 19 11:59:14.812330 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 11:59:14.812330 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application with Lock( May 19 11:59:14.812330 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 11:59:14.812330 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._enter() May 19 11:59:14.812330 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 11:59:14.812330 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 11:59:14.812330 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 11:59:14.812330 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.creator() May 19 11:59:14.812330 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 11:59:14.812330 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application created_value = creator( May 19 11:59:14.813418 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 11:59:14.813418 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 11:59:14.813418 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 11:59:14.813418 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 11:59:14.813418 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 11:59:14.813418 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 11:59:14.813418 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 11:59:14.813418 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application May 19 11:59:14.813418 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 530/1074] 10.222.0.23 () {56 vars in 1111 bytes} [Thu May 19 11:59:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 11:59:14.817954 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f99269-4435-4002-ad2c-de348f195594 None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:59:14.895619 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:59:14.949212 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f99269-4435-4002-ad2c-de348f195594 None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:59:14.955411 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c17468f-b753-4029-aee3-25d9c66afdd2 None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:59:14.993351 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e007c0-ccc8-4531-953d-d9e27d1fea79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:59:14.994087 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e007c0-ccc8-4531-953d-d9e27d1fea79 None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:59:14.994336 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e007c0-ccc8-4531-953d-d9e27d1fea79 None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:59:14.994605 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e007c0-ccc8-4531-953d-d9e27d1fea79 None admin] PATH_INFO: `/v3/projects` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:59:14.995398 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e007c0-ccc8-4531-953d-d9e27d1fea79 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 11:59:14.995704 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e007c0-ccc8-4531-953d-d9e27d1fea79 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:59:14.997170 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e007c0-ccc8-4531-953d-d9e27d1fea79 None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:59:15.001148 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ec546e-df33-4d87-9591-7fd0473684ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:59:15.002157 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ec546e-df33-4d87-9591-7fd0473684ca None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:59:15.002523 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ec546e-df33-4d87-9591-7fd0473684ca None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:59:15.002813 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ec546e-df33-4d87-9591-7fd0473684ca None admin] PATH_INFO: `/v3/users/c1068d87fabf43b2ad94802b55feeb62` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:59:15.004769 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ec546e-df33-4d87-9591-7fd0473684ca None admin] RBAC: Authorizing `identity:get_user(user_id=c1068d87fabf43b2ad94802b55feeb62)` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:59:15.007099 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ec546e-df33-4d87-9591-7fd0473684ca None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:59:15.009735 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 545/1075] 10.222.0.23 () {56 vars in 1177 bytes} [Thu May 19 11:59:14 2022] GET /identity/v3/users/c1068d87fabf43b2ad94802b55feeb62 => generated 285 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 11:59:15.016076 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ec546e-df33-4d87-9591-7fd0473684ca None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:59:15.018610 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 531/1076] 10.222.0.23 () {56 vars in 1148 bytes} [Thu May 19 11:59:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 11:59:15.025073 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e007c0-ccc8-4531-953d-d9e27d1fea79 None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:59:15.050419 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:59:15.140960 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ec546e-df33-4d87-9591-7fd0473684ca None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:59:15.143451 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e007c0-ccc8-4531-953d-d9e27d1fea79 None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:59:15.172180 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:59:15.178042 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae46aa2-f317-4750-8bdc-2be4c21aeed9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:59:15.182976 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae46aa2-f317-4750-8bdc-2be4c21aeed9 None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:59:15.182976 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae46aa2-f317-4750-8bdc-2be4c21aeed9 None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:59:15.182976 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae46aa2-f317-4750-8bdc-2be4c21aeed9 None admin] PATH_INFO: `/v3/projects/545d6fe0ae914bcb891a927c746bd18f` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:59:15.183400 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a65db3f-912f-459a-9105-0e58e0267412 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:59:15.183491 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae46aa2-f317-4750-8bdc-2be4c21aeed9 None admin] RBAC: Authorizing `identity:get_project(project_id=545d6fe0ae914bcb891a927c746bd18f)` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:59:15.183563 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a65db3f-912f-459a-9105-0e58e0267412 None admin] REQUEST_METHOD: `PUT` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:59:15.183563 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a65db3f-912f-459a-9105-0e58e0267412 None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:59:15.183563 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a65db3f-912f-459a-9105-0e58e0267412 None admin] PATH_INFO: `/v3/projects/d70d25bf69314820b8aaecb65576978a/users/34a98d75d84b4c67981377e0f26eca62/roles/ff2dfc5f80304d9cbd3320d8ceffe2a2` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:59:15.183753 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae46aa2-f317-4750-8bdc-2be4c21aeed9 None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:59:15.183753 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 546/1077] 10.222.0.23 () {56 vars in 1186 bytes} [Thu May 19 11:59:15 2022] GET /identity/v3/projects/545d6fe0ae914bcb891a927c746bd18f => generated 311 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:59:15.187139 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a65db3f-912f-459a-9105-0e58e0267412 None admin] RBAC: Authorizing `identity:create_grant(project_id=d70d25bf69314820b8aaecb65576978a, user_id=34a98d75d84b4c67981377e0f26eca62, role_id=ff2dfc5f80304d9cbd3320d8ceffe2a2)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:59:15.187218 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839475ca-a81e-4c54-a71b-d7d984098c67 None None] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:59:15.187218 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839475ca-a81e-4c54-a71b-d7d984098c67 None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:59:15.187218 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839475ca-a81e-4c54-a71b-d7d984098c67 None None] PATH_INFO: `/` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:59:15.187426 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a65db3f-912f-459a-9105-0e58e0267412 None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:59:15.187496 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 547/1078] 10.222.0.23 () {54 vars in 926 bytes} [Thu May 19 11:59:15 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 11:59:15.190310 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839475ca-a81e-4c54-a71b-d7d984098c67 None None] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:59:15.251006 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 532/1079] 10.222.0.23 () {58 vars in 1439 bytes} [Thu May 19 11:59:15 2022] PUT /identity/v3/projects/d70d25bf69314820b8aaecb65576978a/users/34a98d75d84b4c67981377e0f26eca62/roles/ff2dfc5f80304d9cbd3320d8ceffe2a2 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 11:59:15.252391 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0d1549-09e4-41ac-95d3-b9958d754b40 None None] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:59:15.252679 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0d1549-09e4-41ac-95d3-b9958d754b40 None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:59:15.253006 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0d1549-09e4-41ac-95d3-b9958d754b40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:59:15.283431 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.auth.core [None req-ff0d1549-09e4-41ac-95d3-b9958d754b40 None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 11:59:15.284752 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-ff0d1549-09e4-41ac-95d3-b9958d754b40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 11:59:15.326213 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839475ca-a81e-4c54-a71b-d7d984098c67 None None] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:59:15.371223 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36c5bb9-12d2-4dec-871d-4d39b00daf8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:59:15.372427 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36c5bb9-12d2-4dec-871d-4d39b00daf8d None admin] REQUEST_METHOD: `PUT` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:59:15.372731 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36c5bb9-12d2-4dec-871d-4d39b00daf8d None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:59:15.373002 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36c5bb9-12d2-4dec-871d-4d39b00daf8d None admin] PATH_INFO: `/v3/projects/545d6fe0ae914bcb891a927c746bd18f/users/c1068d87fabf43b2ad94802b55feeb62/roles/9db5671c65d3414ea2682afa2ccf1403` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:59:15.375169 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36c5bb9-12d2-4dec-871d-4d39b00daf8d None admin] RBAC: Authorizing `identity:create_grant(project_id=545d6fe0ae914bcb891a927c746bd18f, user_id=c1068d87fabf43b2ad94802b55feeb62, role_id=9db5671c65d3414ea2682afa2ccf1403)` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:59:15.376764 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36c5bb9-12d2-4dec-871d-4d39b00daf8d None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:59:15.379685 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 533/1080] 10.222.0.23 () {58 vars in 1023 bytes} [Thu May 19 11:59:15 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 11:59:15.403719 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0d1549-09e4-41ac-95d3-b9958d754b40 None None] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:59:15.420814 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:59:15.453355 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 548/1081] 10.222.0.23 () {58 vars in 1439 bytes} [Thu May 19 11:59:15 2022] PUT /identity/v3/projects/545d6fe0ae914bcb891a927c746bd18f/users/c1068d87fabf43b2ad94802b55feeb62/roles/9db5671c65d3414ea2682afa2ccf1403 => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 11:59:15.539044 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0d1549-09e4-41ac-95d3-b9958d754b40 None None] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:59:15.586644 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce884c33-12e8-4a08-9268-8e76e4246525 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:59:15.587221 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce884c33-12e8-4a08-9268-8e76e4246525 None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:59:15.587509 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce884c33-12e8-4a08-9268-8e76e4246525 None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:59:15.587883 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce884c33-12e8-4a08-9268-8e76e4246525 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:59:15.673402 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce884c33-12e8-4a08-9268-8e76e4246525 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:59:15.674887 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce884c33-12e8-4a08-9268-8e76e4246525 None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:59:15.688960 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:59:15.789754 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 534/1082] 10.222.0.23 () {58 vars in 1282 bytes} [Thu May 19 11:59:15 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 19 11:59:15.797291 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36c5bb9-12d2-4dec-871d-4d39b00daf8d None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:59:15.861693 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:59:15.906456 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36c5bb9-12d2-4dec-871d-4d39b00daf8d None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:59:15.951687 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0378c2-06eb-4bc6-841f-906f8510bc05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:59:15.952526 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0378c2-06eb-4bc6-841f-906f8510bc05 None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:59:15.952811 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0378c2-06eb-4bc6-841f-906f8510bc05 None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:59:15.953086 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0378c2-06eb-4bc6-841f-906f8510bc05 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:59:15.964488 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0378c2-06eb-4bc6-841f-906f8510bc05 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:59:15.965948 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0378c2-06eb-4bc6-841f-906f8510bc05 None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:59:15.983953 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application [None req-ce0378c2-06eb-4bc6-841f-906f8510bc05 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 11:59:15.983953 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 11:59:15.983953 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 11:59:15.983953 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 11:59:15.983953 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 11:59:15.983953 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 11:59:15.983953 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 11:59:15.983953 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 11:59:15.983953 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 11:59:15.983953 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 11:59:15.983953 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 11:59:15.983953 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 11:59:15.983953 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 11:59:15.983953 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 11:59:15.983953 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 11:59:15.984911 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 11:59:15.984911 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 11:59:15.984911 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 11:59:15.984911 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 11:59:15.984911 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 11:59:15.984911 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 11:59:15.984911 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.get_or_create( May 19 11:59:15.984911 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 11:59:15.984911 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application with Lock( May 19 11:59:15.984911 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 11:59:15.984911 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._enter() May 19 11:59:15.984911 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 11:59:15.984911 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 11:59:15.984911 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 11:59:15.984911 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.creator() May 19 11:59:15.984911 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 11:59:15.984911 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application created_value = creator( May 19 11:59:15.986268 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 11:59:15.986268 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 11:59:15.986268 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 11:59:15.986268 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application May 19 11:59:15.986268 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 549/1083] 10.222.0.23 () {56 vars in 1108 bytes} [Thu May 19 11:59:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 11:59:15.990990 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce884c33-12e8-4a08-9268-8e76e4246525 None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:59:16.094092 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce884c33-12e8-4a08-9268-8e76e4246525 None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:59:16.105180 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:59:16.129851 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638a160a-b294-481c-b050-f8bcdb57b732 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:59:16.130435 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638a160a-b294-481c-b050-f8bcdb57b732 None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:59:16.130714 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638a160a-b294-481c-b050-f8bcdb57b732 None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:59:16.130987 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638a160a-b294-481c-b050-f8bcdb57b732 None admin] PATH_INFO: `/v3/domains` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:59:16.131713 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638a160a-b294-481c-b050-f8bcdb57b732 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 11:59:16.132049 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638a160a-b294-481c-b050-f8bcdb57b732 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:59:16.133528 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638a160a-b294-481c-b050-f8bcdb57b732 None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:59:16.155840 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 535/1084] 10.222.0.23 () {56 vars in 1109 bytes} [Thu May 19 11:59:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 11:59:17.539245 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abded692-daf0-4680-999c-b82061746e85 None None] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:59:17.539821 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abded692-daf0-4680-999c-b82061746e85 None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:59:17.540042 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abded692-daf0-4680-999c-b82061746e85 None None] PATH_INFO: `/` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:59:17.541382 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 550/1085] 10.222.0.23 () {54 vars in 926 bytes} [Thu May 19 11:59:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 11:59:17.548625 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35d4729-e313-4b7e-b5dc-d21dd744cc8c None None] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:59:17.549166 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35d4729-e313-4b7e-b5dc-d21dd744cc8c None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:59:17.549166 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35d4729-e313-4b7e-b5dc-d21dd744cc8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:59:17.567395 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.auth.core [None req-f35d4729-e313-4b7e-b5dc-d21dd744cc8c None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 11:59:17.568609 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-f35d4729-e313-4b7e-b5dc-d21dd744cc8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 11:59:17.637145 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 536/1086] 10.222.0.23 () {58 vars in 1023 bytes} [Thu May 19 11:59:17 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 11:59:17.660470 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abded692-daf0-4680-999c-b82061746e85 None None] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:59:17.714348 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b2b976-eee9-40e7-9589-ce50dc622952 None None] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:59:17.714657 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b2b976-eee9-40e7-9589-ce50dc622952 None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:59:17.714941 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b2b976-eee9-40e7-9589-ce50dc622952 None None] PATH_INFO: `/` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:59:17.716407 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 537/1087] 10.222.0.23 () {54 vars in 926 bytes} [Thu May 19 11:59:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 11:59:17.721841 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919b98e2-1713-4ec9-a0ff-e92e78b5253d None None] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:59:17.722125 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919b98e2-1713-4ec9-a0ff-e92e78b5253d None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:59:17.722408 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919b98e2-1713-4ec9-a0ff-e92e78b5253d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:59:17.740471 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.auth.core [None req-919b98e2-1713-4ec9-a0ff-e92e78b5253d None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 11:59:17.741669 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-919b98e2-1713-4ec9-a0ff-e92e78b5253d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 11:59:17.765470 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abded692-daf0-4680-999c-b82061746e85 None None] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:59:17.776724 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:59:17.801276 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83adc993-63fb-4e20-90e9-f6cb0816782c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:59:17.802017 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83adc993-63fb-4e20-90e9-f6cb0816782c None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:59:17.802299 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83adc993-63fb-4e20-90e9-f6cb0816782c None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:59:17.802572 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83adc993-63fb-4e20-90e9-f6cb0816782c None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:59:17.809645 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 538/1088] 10.222.0.23 () {58 vars in 1023 bytes} [Thu May 19 11:59:17 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 11:59:17.813708 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83adc993-63fb-4e20-90e9-f6cb0816782c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 11:59:17.822872 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application [None req-83adc993-63fb-4e20-90e9-f6cb0816782c None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 19 11:59:17.822872 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 11:59:17.822872 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 11:59:17.822872 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 11:59:17.822872 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 11:59:17.822872 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 11:59:17.822872 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 11:59:17.822872 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 11:59:17.822872 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 11:59:17.822872 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 11:59:17.822872 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 11:59:17.822872 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 11:59:17.822872 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 11:59:17.822872 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 11:59:17.822872 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 11:59:17.822872 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 11:59:17.824042 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 11:59:17.824042 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 11:59:17.824042 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 11:59:17.824042 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 11:59:17.824042 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 11:59:17.824042 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 11:59:17.824042 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 11:59:17.824042 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 11:59:17.824042 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 11:59:17.824042 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.get_or_create( May 19 11:59:17.824042 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 11:59:17.824042 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application with Lock( May 19 11:59:17.824042 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 11:59:17.824042 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._enter() May 19 11:59:17.824042 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 11:59:17.824042 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 11:59:17.824888 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 11:59:17.824888 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.creator() May 19 11:59:17.824888 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 11:59:17.824888 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application created_value = creator( May 19 11:59:17.824888 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 11:59:17.824888 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 11:59:17.824888 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 11:59:17.824888 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 11:59:17.824888 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 11:59:17.824888 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 11:59:17.824888 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 19 11:59:17.824888 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application May 19 11:59:17.828087 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 551/1089] 10.222.0.23 () {56 vars in 1120 bytes} [Thu May 19 11:59:17 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 11:59:17.830302 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919b98e2-1713-4ec9-a0ff-e92e78b5253d None None] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:59:17.834344 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83adc993-63fb-4e20-90e9-f6cb0816782c None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:59:17.862977 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:59:17.937328 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83adc993-63fb-4e20-90e9-f6cb0816782c None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:59:17.939822 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919b98e2-1713-4ec9-a0ff-e92e78b5253d None None] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:59:17.977615 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1d506a-baaf-47bb-92c3-6d2ce80e3433 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:59:17.978293 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1d506a-baaf-47bb-92c3-6d2ce80e3433 None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:59:17.978584 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1d506a-baaf-47bb-92c3-6d2ce80e3433 None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:59:17.978951 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1d506a-baaf-47bb-92c3-6d2ce80e3433 None admin] PATH_INFO: `/v3/roles` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:59:17.979833 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1d506a-baaf-47bb-92c3-6d2ce80e3433 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 11:59:17.980297 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1d506a-baaf-47bb-92c3-6d2ce80e3433 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:59:17.981694 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1d506a-baaf-47bb-92c3-6d2ce80e3433 None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:59:17.986898 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e4ceab-6065-4c10-8e66-c4c85d9fa726 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:59:17.987711 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e4ceab-6065-4c10-8e66-c4c85d9fa726 None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:59:17.988024 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e4ceab-6065-4c10-8e66-c4c85d9fa726 None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:59:17.988311 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e4ceab-6065-4c10-8e66-c4c85d9fa726 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:59:17.996069 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 539/1090] 10.222.0.23 () {56 vars in 1115 bytes} [Thu May 19 11:59:17 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 11:59:17.997002 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1d506a-baaf-47bb-92c3-6d2ce80e3433 None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:59:18.000379 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e4ceab-6065-4c10-8e66-c4c85d9fa726 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 11:59:18.008385 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application [None req-19e4ceab-6065-4c10-8e66-c4c85d9fa726 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 11:59:18.008385 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 11:59:18.008385 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 11:59:18.008385 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 11:59:18.008385 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 11:59:18.008385 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 11:59:18.008385 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 11:59:18.008385 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 11:59:18.008385 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 11:59:18.008385 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 11:59:18.008385 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 11:59:18.008385 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 11:59:18.008385 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 11:59:18.008385 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 11:59:18.008385 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 11:59:18.008385 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 11:59:18.009355 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 11:59:18.009355 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 11:59:18.009355 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 11:59:18.009355 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 11:59:18.009355 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 11:59:18.009355 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 11:59:18.009355 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 11:59:18.009355 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 11:59:18.009355 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 11:59:18.009355 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.get_or_create( May 19 11:59:18.009355 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 11:59:18.009355 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application with Lock( May 19 11:59:18.009355 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 11:59:18.009355 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._enter() May 19 11:59:18.009355 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 11:59:18.009355 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 11:59:18.010225 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 11:59:18.010225 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.creator() May 19 11:59:18.010225 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 11:59:18.010225 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application created_value = creator( May 19 11:59:18.010225 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 11:59:18.010225 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 11:59:18.010225 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 11:59:18.010225 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 11:59:18.010225 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 11:59:18.010225 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 11:59:18.010225 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 11:59:18.010225 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application May 19 11:59:18.013010 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 552/1091] 10.222.0.23 () {56 vars in 1096 bytes} [Thu May 19 11:59:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 11:59:18.019412 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e4ceab-6065-4c10-8e66-c4c85d9fa726 None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:59:18.061075 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:59:18.109787 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:59:18.118272 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1d506a-baaf-47bb-92c3-6d2ce80e3433 None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:59:18.142676 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e4ceab-6065-4c10-8e66-c4c85d9fa726 None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:59:18.153459 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb76ca4-e4ef-4147-9eb3-99493cc2db05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:59:18.155137 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb76ca4-e4ef-4147-9eb3-99493cc2db05 None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:59:18.155427 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb76ca4-e4ef-4147-9eb3-99493cc2db05 None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:59:18.155705 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb76ca4-e4ef-4147-9eb3-99493cc2db05 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:59:18.166754 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb76ca4-e4ef-4147-9eb3-99493cc2db05 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:59:18.168519 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb76ca4-e4ef-4147-9eb3-99493cc2db05 None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:59:18.179992 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5620f3f-aaef-4fc8-a339-f5639622c5e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:59:18.180673 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5620f3f-aaef-4fc8-a339-f5639622c5e4 None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:59:18.180964 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5620f3f-aaef-4fc8-a339-f5639622c5e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:59:18.181248 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5620f3f-aaef-4fc8-a339-f5639622c5e4 None admin] PATH_INFO: `/v3/roles` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:59:18.182002 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5620f3f-aaef-4fc8-a339-f5639622c5e4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 11:59:18.182315 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5620f3f-aaef-4fc8-a339-f5639622c5e4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:59:18.183726 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5620f3f-aaef-4fc8-a339-f5639622c5e4 None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:59:18.188016 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application [None req-aeb76ca4-e4ef-4147-9eb3-99493cc2db05 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 11:59:18.188016 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 11:59:18.188016 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 11:59:18.188016 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 11:59:18.188016 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 11:59:18.188016 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 11:59:18.188016 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 11:59:18.188016 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 11:59:18.188016 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 11:59:18.188016 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 11:59:18.188016 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 11:59:18.188016 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 11:59:18.188016 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 11:59:18.188016 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 11:59:18.188016 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 11:59:18.189052 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 11:59:18.189052 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 11:59:18.189052 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 11:59:18.189052 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 11:59:18.189052 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 11:59:18.189052 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 11:59:18.189052 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.get_or_create( May 19 11:59:18.189052 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 11:59:18.189052 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application with Lock( May 19 11:59:18.189052 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 11:59:18.189052 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._enter() May 19 11:59:18.189052 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 11:59:18.189052 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 11:59:18.189052 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 11:59:18.189052 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.creator() May 19 11:59:18.189052 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 11:59:18.189052 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application created_value = creator( May 19 11:59:18.189972 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 11:59:18.189972 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 11:59:18.189972 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 11:59:18.189972 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application May 19 11:59:18.195834 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 553/1092] 10.222.0.23 () {56 vars in 1099 bytes} [Thu May 19 11:59:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:59:18.196021 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 540/1093] 10.222.0.23 () {56 vars in 1108 bytes} [Thu May 19 11:59:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 11:59:18.196121 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5620f3f-aaef-4fc8-a339-f5639622c5e4 None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:59:18.198407 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb76ca4-e4ef-4147-9eb3-99493cc2db05 None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:59:18.295350 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:59:18.304307 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5620f3f-aaef-4fc8-a339-f5639622c5e4 None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:59:18.306133 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb76ca4-e4ef-4147-9eb3-99493cc2db05 None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:59:18.338968 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17807b72-1c18-4812-8f51-0608f60e13b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:59:18.339555 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17807b72-1c18-4812-8f51-0608f60e13b0 None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:59:18.339876 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17807b72-1c18-4812-8f51-0608f60e13b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:59:18.340154 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17807b72-1c18-4812-8f51-0608f60e13b0 None admin] PATH_INFO: `/v3/domains` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:59:18.340698 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca04e649-6eaf-497f-a921-5a26f057920b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:59:18.340875 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17807b72-1c18-4812-8f51-0608f60e13b0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 11:59:18.341319 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17807b72-1c18-4812-8f51-0608f60e13b0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:59:18.343028 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca04e649-6eaf-497f-a921-5a26f057920b None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:59:18.343323 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca04e649-6eaf-497f-a921-5a26f057920b None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:59:18.343424 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17807b72-1c18-4812-8f51-0608f60e13b0 None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:59:18.343585 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca04e649-6eaf-497f-a921-5a26f057920b None admin] PATH_INFO: `/v3/users/c1068d87fabf43b2ad94802b55feeb62` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:59:18.344868 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca04e649-6eaf-497f-a921-5a26f057920b None admin] RBAC: Authorizing `identity:get_user(user_id=c1068d87fabf43b2ad94802b55feeb62)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:59:18.346307 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca04e649-6eaf-497f-a921-5a26f057920b None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:59:18.348015 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 541/1094] 10.222.0.23 () {56 vars in 1177 bytes} [Thu May 19 11:59:18 2022] GET /identity/v3/users/c1068d87fabf43b2ad94802b55feeb62 => generated 285 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 11:59:18.353989 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca04e649-6eaf-497f-a921-5a26f057920b None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:59:18.360341 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 554/1095] 10.222.0.23 () {56 vars in 1109 bytes} [Thu May 19 11:59:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 11:59:18.361375 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17807b72-1c18-4812-8f51-0608f60e13b0 None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:59:18.369310 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:59:18.457398 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca04e649-6eaf-497f-a921-5a26f057920b None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:59:18.463229 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17807b72-1c18-4812-8f51-0608f60e13b0 None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:59:18.491437 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd49706-72a1-4c91-9499-763282f61d3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:59:18.492124 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd49706-72a1-4c91-9499-763282f61d3d None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:59:18.492411 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd49706-72a1-4c91-9499-763282f61d3d None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:59:18.492688 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd49706-72a1-4c91-9499-763282f61d3d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:59:18.498483 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec351db4-9db6-4dd4-a343-20db64047a49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:59:18.499281 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec351db4-9db6-4dd4-a343-20db64047a49 None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:59:18.499570 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec351db4-9db6-4dd4-a343-20db64047a49 None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:59:18.499884 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec351db4-9db6-4dd4-a343-20db64047a49 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:59:18.507513 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec351db4-9db6-4dd4-a343-20db64047a49 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:59:18.509015 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec351db4-9db6-4dd4-a343-20db64047a49 None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:59:18.523087 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application [None req-ec351db4-9db6-4dd4-a343-20db64047a49 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 19 11:59:18.523087 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 11:59:18.523087 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 11:59:18.523087 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 11:59:18.523087 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 11:59:18.523087 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 11:59:18.523087 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 11:59:18.523087 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 11:59:18.523087 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 11:59:18.523087 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 11:59:18.523087 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 11:59:18.523087 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 11:59:18.523087 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 11:59:18.523087 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 11:59:18.523087 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 11:59:18.523087 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 11:59:18.524276 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 11:59:18.524276 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 11:59:18.524276 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 11:59:18.524276 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 11:59:18.524276 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 11:59:18.524276 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 11:59:18.524276 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 11:59:18.524276 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 11:59:18.524276 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 11:59:18.524276 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.get_or_create( May 19 11:59:18.524276 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 11:59:18.524276 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application with Lock( May 19 11:59:18.524276 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 11:59:18.524276 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._enter() May 19 11:59:18.524276 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 11:59:18.524276 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 11:59:18.524276 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 11:59:18.524276 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.creator() May 19 11:59:18.525453 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 11:59:18.525453 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application created_value = creator( May 19 11:59:18.525453 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 11:59:18.525453 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 11:59:18.525453 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 11:59:18.525453 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 11:59:18.525453 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 11:59:18.525453 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 11:59:18.525453 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 19 11:59:18.525453 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application May 19 11:59:18.525453 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 555/1096] 10.222.0.23 () {56 vars in 1093 bytes} [Thu May 19 11:59:18 2022] GET /identity/v3/users/nova => generated 82 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 11:59:18.530194 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec351db4-9db6-4dd4-a343-20db64047a49 None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:59:18.545669 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:59:18.575018 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd49706-72a1-4c91-9499-763282f61d3d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:59:18.577159 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd49706-72a1-4c91-9499-763282f61d3d None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:59:18.590957 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:59:18.638486 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec351db4-9db6-4dd4-a343-20db64047a49 None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:59:18.673922 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4e5d17-3ca4-4bc6-94c9-50b5873c3a04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:59:18.674595 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4e5d17-3ca4-4bc6-94c9-50b5873c3a04 None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:59:18.674879 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4e5d17-3ca4-4bc6-94c9-50b5873c3a04 None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:59:18.675157 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4e5d17-3ca4-4bc6-94c9-50b5873c3a04 None admin] PATH_INFO: `/v3/users` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:59:18.676147 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4e5d17-3ca4-4bc6-94c9-50b5873c3a04 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 11:59:18.676462 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4e5d17-3ca4-4bc6-94c9-50b5873c3a04 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:59:18.677903 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 542/1097] 10.222.0.23 () {58 vars in 1282 bytes} [Thu May 19 11:59:18 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 19 11:59:18.678008 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4e5d17-3ca4-4bc6-94c9-50b5873c3a04 None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:59:18.683837 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd49706-72a1-4c91-9499-763282f61d3d None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:59:18.696009 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 556/1098] 10.222.0.23 () {56 vars in 1133 bytes} [Thu May 19 11:59:18 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 11:59:18.702388 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4e5d17-3ca4-4bc6-94c9-50b5873c3a04 None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:59:18.772305 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:59:18.782929 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd49706-72a1-4c91-9499-763282f61d3d None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:59:18.796869 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:59:18.803231 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4e5d17-3ca4-4bc6-94c9-50b5873c3a04 None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:59:18.817763 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1fa540-faab-45dc-9845-a5c3b5fd93a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:59:18.818625 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1fa540-faab-45dc-9845-a5c3b5fd93a1 None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:59:18.818902 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1fa540-faab-45dc-9845-a5c3b5fd93a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:59:18.819176 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1fa540-faab-45dc-9845-a5c3b5fd93a1 None admin] PATH_INFO: `/v3/projects/545d6fe0ae914bcb891a927c746bd18f` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:59:18.820417 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b1fa540-faab-45dc-9845-a5c3b5fd93a1 None admin] RBAC: Authorizing `identity:get_project(project_id=545d6fe0ae914bcb891a927c746bd18f)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:59:18.822108 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b1fa540-faab-45dc-9845-a5c3b5fd93a1 None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:59:18.823779 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 543/1099] 10.222.0.23 () {56 vars in 1186 bytes} [Thu May 19 11:59:18 2022] GET /identity/v3/projects/545d6fe0ae914bcb891a927c746bd18f => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 11:59:18.829745 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1fa540-faab-45dc-9845-a5c3b5fd93a1 None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:59:18.837658 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae79485a-f7fd-4b37-b56d-ea21688ac611 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:59:18.838318 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae79485a-f7fd-4b37-b56d-ea21688ac611 None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:59:18.838608 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae79485a-f7fd-4b37-b56d-ea21688ac611 None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:59:18.838889 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae79485a-f7fd-4b37-b56d-ea21688ac611 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:59:18.910693 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae79485a-f7fd-4b37-b56d-ea21688ac611 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:59:18.912135 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae79485a-f7fd-4b37-b56d-ea21688ac611 None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:59:18.931162 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1fa540-faab-45dc-9845-a5c3b5fd93a1 None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:59:18.967594 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14dda6fe-f92a-4a8e-819f-959b971288cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:59:18.968276 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14dda6fe-f92a-4a8e-819f-959b971288cd None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:59:18.968560 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14dda6fe-f92a-4a8e-819f-959b971288cd None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:59:18.968836 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14dda6fe-f92a-4a8e-819f-959b971288cd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:59:18.969667 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14dda6fe-f92a-4a8e-819f-959b971288cd None admin] RBAC: Adding query filter params (role.id=9db5671c65d3414ea2682afa2ccf1403, scope.project.id=545d6fe0ae914bcb891a927c746bd18f, user.id=c1068d87fabf43b2ad94802b55feeb62) {{(pid=80539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 11:59:18.969980 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14dda6fe-f92a-4a8e-819f-959b971288cd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:59:18.971409 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14dda6fe-f92a-4a8e-819f-959b971288cd None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:59:18.986829 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 544/1100] 10.222.0.23 () {56 vars in 1374 bytes} [Thu May 19 11:59:18 2022] GET /identity/v3/role_assignments?user.id=c1068d87fabf43b2ad94802b55feeb62&scope.project.id=545d6fe0ae914bcb891a927c746bd18f&role.id=9db5671c65d3414ea2682afa2ccf1403 => generated 608 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 11:59:18.987090 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:59:19.022670 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 557/1101] 10.222.0.23 () {58 vars in 1282 bytes} [Thu May 19 11:59:18 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 19 11:59:19.028635 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14dda6fe-f92a-4a8e-819f-959b971288cd None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:59:19.079628 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:59:19.139316 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14dda6fe-f92a-4a8e-819f-959b971288cd None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:59:19.175768 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150619a7-5433-4329-aa56-b2869499f8ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:59:19.176514 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150619a7-5433-4329-aa56-b2869499f8ea None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:59:19.176794 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150619a7-5433-4329-aa56-b2869499f8ea None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:59:19.177067 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150619a7-5433-4329-aa56-b2869499f8ea None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:59:19.187335 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-150619a7-5433-4329-aa56-b2869499f8ea None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:59:19.188832 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-150619a7-5433-4329-aa56-b2869499f8ea None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:59:19.206901 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application [None req-150619a7-5433-4329-aa56-b2869499f8ea None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 11:59:19.206901 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 11:59:19.206901 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 11:59:19.206901 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 11:59:19.206901 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 11:59:19.206901 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 11:59:19.206901 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 11:59:19.206901 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 11:59:19.206901 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 11:59:19.206901 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 11:59:19.206901 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 11:59:19.206901 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 11:59:19.206901 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 11:59:19.206901 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 11:59:19.206901 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 11:59:19.207877 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 11:59:19.207877 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 11:59:19.207877 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 11:59:19.207877 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 11:59:19.207877 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 11:59:19.207877 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 11:59:19.207877 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.get_or_create( May 19 11:59:19.207877 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 11:59:19.207877 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application with Lock( May 19 11:59:19.207877 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 11:59:19.207877 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._enter() May 19 11:59:19.207877 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 11:59:19.207877 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 11:59:19.207877 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 11:59:19.207877 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.creator() May 19 11:59:19.207877 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 11:59:19.207877 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application created_value = creator( May 19 11:59:19.208776 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 11:59:19.208776 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 11:59:19.208776 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 11:59:19.208776 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application May 19 11:59:19.208776 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 545/1102] 10.222.0.23 () {56 vars in 1108 bytes} [Thu May 19 11:59:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 11:59:19.213894 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae79485a-f7fd-4b37-b56d-ea21688ac611 None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:59:19.313243 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae79485a-f7fd-4b37-b56d-ea21688ac611 None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:59:19.348574 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9358e8cf-e1d1-4479-af24-e5aa8d5b6876 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:59:19.349238 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9358e8cf-e1d1-4479-af24-e5aa8d5b6876 None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:59:19.349526 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9358e8cf-e1d1-4479-af24-e5aa8d5b6876 None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:59:19.349811 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9358e8cf-e1d1-4479-af24-e5aa8d5b6876 None admin] PATH_INFO: `/v3/domains` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:59:19.350535 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9358e8cf-e1d1-4479-af24-e5aa8d5b6876 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 11:59:19.350847 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9358e8cf-e1d1-4479-af24-e5aa8d5b6876 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:59:19.352283 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9358e8cf-e1d1-4479-af24-e5aa8d5b6876 None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:59:19.363938 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 558/1103] 10.222.0.23 () {56 vars in 1109 bytes} [Thu May 19 11:59:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:59:19.369441 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150619a7-5433-4329-aa56-b2869499f8ea None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:59:19.468304 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:59:19.473863 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150619a7-5433-4329-aa56-b2869499f8ea None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:59:19.508731 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9eed61-6ae1-4888-9017-103cabee2e0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:59:19.509519 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9eed61-6ae1-4888-9017-103cabee2e0f None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:59:19.509810 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9eed61-6ae1-4888-9017-103cabee2e0f None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:59:19.510085 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9eed61-6ae1-4888-9017-103cabee2e0f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:59:19.517427 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9eed61-6ae1-4888-9017-103cabee2e0f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:59:19.518880 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9eed61-6ae1-4888-9017-103cabee2e0f None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:59:19.533371 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application [None req-0a9eed61-6ae1-4888-9017-103cabee2e0f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 11:59:19.533371 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 11:59:19.533371 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 11:59:19.533371 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 11:59:19.533371 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 11:59:19.533371 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 11:59:19.533371 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 11:59:19.533371 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 11:59:19.533371 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 11:59:19.533371 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 11:59:19.533371 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 11:59:19.533371 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 11:59:19.533371 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 11:59:19.533371 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 11:59:19.533371 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 11:59:19.534321 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 11:59:19.534321 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 11:59:19.534321 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 11:59:19.534321 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 11:59:19.534321 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 11:59:19.534321 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 11:59:19.534321 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.get_or_create( May 19 11:59:19.534321 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 11:59:19.534321 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application with Lock( May 19 11:59:19.534321 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 11:59:19.534321 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._enter() May 19 11:59:19.534321 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 11:59:19.534321 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 11:59:19.534321 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 11:59:19.534321 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.creator() May 19 11:59:19.534321 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 11:59:19.534321 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application created_value = creator( May 19 11:59:19.535198 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 11:59:19.535198 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 11:59:19.535198 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 11:59:19.535198 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 11:59:19.535198 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 11:59:19.535198 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 11:59:19.535198 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 11:59:19.535198 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application May 19 11:59:19.535198 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 546/1104] 10.222.0.23 () {56 vars in 1111 bytes} [Thu May 19 11:59:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 11:59:19.540424 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9358e8cf-e1d1-4479-af24-e5aa8d5b6876 None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:59:19.571242 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:59:19.643304 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9358e8cf-e1d1-4479-af24-e5aa8d5b6876 None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:59:19.678716 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab19f16b-9491-4652-ab15-0d42296f05e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:59:19.679368 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab19f16b-9491-4652-ab15-0d42296f05e5 None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:59:19.679648 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab19f16b-9491-4652-ab15-0d42296f05e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:59:19.679952 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab19f16b-9491-4652-ab15-0d42296f05e5 None admin] PATH_INFO: `/v3/projects` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:59:19.680732 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab19f16b-9491-4652-ab15-0d42296f05e5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 11:59:19.681063 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab19f16b-9491-4652-ab15-0d42296f05e5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:59:19.682483 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab19f16b-9491-4652-ab15-0d42296f05e5 None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:59:19.694332 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 559/1105] 10.222.0.23 () {56 vars in 1148 bytes} [Thu May 19 11:59:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:59:19.701052 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9eed61-6ae1-4888-9017-103cabee2e0f None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:59:19.801714 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9eed61-6ae1-4888-9017-103cabee2e0f None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:59:19.835939 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615e99bf-0f1a-4695-9a5c-3efb3863b29a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:59:19.836595 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615e99bf-0f1a-4695-9a5c-3efb3863b29a None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:59:19.836975 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615e99bf-0f1a-4695-9a5c-3efb3863b29a None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:59:19.837308 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615e99bf-0f1a-4695-9a5c-3efb3863b29a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:59:19.838152 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615e99bf-0f1a-4695-9a5c-3efb3863b29a None admin] RBAC: Adding query filter params (role.id=ff2dfc5f80304d9cbd3320d8ceffe2a2, scope.project.id=d70d25bf69314820b8aaecb65576978a, user.id=34a98d75d84b4c67981377e0f26eca62) {{(pid=80539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 11:59:19.838468 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615e99bf-0f1a-4695-9a5c-3efb3863b29a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:59:19.839933 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615e99bf-0f1a-4695-9a5c-3efb3863b29a None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:59:19.846780 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 547/1106] 10.222.0.23 () {56 vars in 1374 bytes} [Thu May 19 11:59:19 2022] GET /identity/v3/role_assignments?user.id=34a98d75d84b4c67981377e0f26eca62&scope.project.id=d70d25bf69314820b8aaecb65576978a&role.id=ff2dfc5f80304d9cbd3320d8ceffe2a2 => generated 608 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:59:21.266297 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cab29cb-87fa-4806-b4ff-a40df4b453b7 None None] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:59:21.266827 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cab29cb-87fa-4806-b4ff-a40df4b453b7 None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:59:21.266827 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cab29cb-87fa-4806-b4ff-a40df4b453b7 None None] PATH_INFO: `/` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:59:21.268303 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 560/1107] 10.222.0.23 () {54 vars in 926 bytes} [Thu May 19 11:59:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 11:59:21.274165 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe795fcc-77d0-461d-a97b-379ddf007e71 None None] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:59:21.274660 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe795fcc-77d0-461d-a97b-379ddf007e71 None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:59:21.274750 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe795fcc-77d0-461d-a97b-379ddf007e71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:59:21.293424 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.auth.core [None req-fe795fcc-77d0-461d-a97b-379ddf007e71 None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 11:59:21.294693 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-fe795fcc-77d0-461d-a97b-379ddf007e71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 11:59:21.309331 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:59:21.364250 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 548/1108] 10.222.0.23 () {58 vars in 1023 bytes} [Thu May 19 11:59:21 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 11:59:21.389608 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cab29cb-87fa-4806-b4ff-a40df4b453b7 None None] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:59:21.488280 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:59:21.498959 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cab29cb-87fa-4806-b4ff-a40df4b453b7 None None] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:59:21.534663 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f238fc91-61da-4426-a9aa-48a67a532be1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:59:21.535408 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f238fc91-61da-4426-a9aa-48a67a532be1 None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:59:21.535691 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f238fc91-61da-4426-a9aa-48a67a532be1 None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:59:21.535996 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f238fc91-61da-4426-a9aa-48a67a532be1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:59:21.537198 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f238fc91-61da-4426-a9aa-48a67a532be1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:59:21.538654 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f238fc91-61da-4426-a9aa-48a67a532be1 None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:59:21.540328 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 561/1109] 10.222.0.23 () {56 vars in 1108 bytes} [Thu May 19 11:59:21 2022] GET /identity/v3/domains/default => generated 199 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:59:21.546655 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe795fcc-77d0-461d-a97b-379ddf007e71 None None] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:59:21.647430 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe795fcc-77d0-461d-a97b-379ddf007e71 None None] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:59:21.683156 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba3f826-5aaf-4ea0-9b78-dd2846e5ddfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:59:21.683848 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba3f826-5aaf-4ea0-9b78-dd2846e5ddfa None admin] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:59:21.684134 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba3f826-5aaf-4ea0-9b78-dd2846e5ddfa None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:59:21.684409 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba3f826-5aaf-4ea0-9b78-dd2846e5ddfa None admin] PATH_INFO: `/v3/users` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:59:21.685241 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba3f826-5aaf-4ea0-9b78-dd2846e5ddfa None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:59:21.686946 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba3f826-5aaf-4ea0-9b78-dd2846e5ddfa None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:59:21.765872 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 549/1110] 10.222.0.23 () {60 vars in 1132 bytes} [Thu May 19 11:59:21 2022] POST /identity/v3/users => generated 286 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 11:59:24.033705 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f0a91f-f11e-4b4b-ac2b-6833baea808d None None] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:59:24.034220 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f0a91f-f11e-4b4b-ac2b-6833baea808d None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:59:24.034376 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f0a91f-f11e-4b4b-ac2b-6833baea808d None None] PATH_INFO: `/` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:59:24.035763 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 562/1111] 10.222.0.23 () {54 vars in 926 bytes} [Thu May 19 11:59:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 11:59:24.040918 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3a4541-cc3c-4818-b2e9-f8c2a2445ac0 None None] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:59:24.041318 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3a4541-cc3c-4818-b2e9-f8c2a2445ac0 None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:59:24.041477 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3a4541-cc3c-4818-b2e9-f8c2a2445ac0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:59:24.053426 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:59:24.058692 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.auth.core [None req-0a3a4541-cc3c-4818-b2e9-f8c2a2445ac0 None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 11:59:24.059991 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-0a3a4541-cc3c-4818-b2e9-f8c2a2445ac0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 11:59:24.125981 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 550/1112] 10.222.0.23 () {58 vars in 1023 bytes} [Thu May 19 11:59:24 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 11:59:24.150527 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f0a91f-f11e-4b4b-ac2b-6833baea808d None None] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:59:24.253335 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f0a91f-f11e-4b4b-ac2b-6833baea808d None None] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:59:24.288670 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1182ca78-2101-42e9-a5ca-665bae032011 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:59:24.289579 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1182ca78-2101-42e9-a5ca-665bae032011 None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:59:24.289820 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1182ca78-2101-42e9-a5ca-665bae032011 None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:59:24.290098 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1182ca78-2101-42e9-a5ca-665bae032011 None admin] PATH_INFO: `/v3/roles/8215583bb4224c75b093ae8e3993eb9f` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:59:24.291580 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1182ca78-2101-42e9-a5ca-665bae032011 None admin] RBAC: Authorizing `identity:get_role(role_id=8215583bb4224c75b093ae8e3993eb9f)` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:59:24.293023 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1182ca78-2101-42e9-a5ca-665bae032011 None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:59:24.294476 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 563/1113] 10.222.0.23 () {56 vars in 1177 bytes} [Thu May 19 11:59:24 2022] GET /identity/v3/roles/8215583bb4224c75b093ae8e3993eb9f => generated 223 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:59:24.300431 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3a4541-cc3c-4818-b2e9-f8c2a2445ac0 None None] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:59:24.396405 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3a4541-cc3c-4818-b2e9-f8c2a2445ac0 None None] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:59:24.429223 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec89dee0-584d-4596-a6ba-4e4a9c9ce750 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:59:24.430093 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec89dee0-584d-4596-a6ba-4e4a9c9ce750 None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:59:24.430370 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec89dee0-584d-4596-a6ba-4e4a9c9ce750 None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:59:24.430638 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec89dee0-584d-4596-a6ba-4e4a9c9ce750 None admin] PATH_INFO: `/v3/users/4c68ca93b40941abb187e85c18e2f8fb` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:59:24.433428 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:59:24.444402 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec89dee0-584d-4596-a6ba-4e4a9c9ce750 None admin] RBAC: Authorizing `identity:get_user(user_id=4c68ca93b40941abb187e85c18e2f8fb)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:59:24.445850 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec89dee0-584d-4596-a6ba-4e4a9c9ce750 None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:59:24.448123 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 551/1114] 10.222.0.23 () {56 vars in 1177 bytes} [Thu May 19 11:59:24 2022] GET /identity/v3/users/4c68ca93b40941abb187e85c18e2f8fb => generated 286 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:59:24.455065 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1182ca78-2101-42e9-a5ca-665bae032011 None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:59:24.483324 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:59:24.555568 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1182ca78-2101-42e9-a5ca-665bae032011 None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:59:24.588875 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53eba135-d440-4812-976f-453c8cf4ee4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:59:24.589519 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53eba135-d440-4812-976f-453c8cf4ee4b None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:59:24.589803 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53eba135-d440-4812-976f-453c8cf4ee4b None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:59:24.590079 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53eba135-d440-4812-976f-453c8cf4ee4b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:59:24.657207 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53eba135-d440-4812-976f-453c8cf4ee4b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:59:24.658660 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53eba135-d440-4812-976f-453c8cf4ee4b None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:59:24.683839 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:59:24.758242 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 564/1115] 10.222.0.23 () {58 vars in 1282 bytes} [Thu May 19 11:59:24 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 11:59:24.764966 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec89dee0-584d-4596-a6ba-4e4a9c9ce750 None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:59:24.863035 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec89dee0-584d-4596-a6ba-4e4a9c9ce750 None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:59:24.897396 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95bdab37-fd92-411a-b50b-157f0326e89f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:59:24.898244 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bdab37-fd92-411a-b50b-157f0326e89f None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:59:24.898551 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bdab37-fd92-411a-b50b-157f0326e89f None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:59:24.898827 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bdab37-fd92-411a-b50b-157f0326e89f None admin] PATH_INFO: `/v3/projects/545d6fe0ae914bcb891a927c746bd18f` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:59:24.900032 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95bdab37-fd92-411a-b50b-157f0326e89f None admin] RBAC: Authorizing `identity:get_project(project_id=545d6fe0ae914bcb891a927c746bd18f)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:59:24.901477 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95bdab37-fd92-411a-b50b-157f0326e89f None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:59:24.903170 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 552/1116] 10.222.0.23 () {56 vars in 1186 bytes} [Thu May 19 11:59:24 2022] GET /identity/v3/projects/545d6fe0ae914bcb891a927c746bd18f => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:59:24.909444 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53eba135-d440-4812-976f-453c8cf4ee4b None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:59:25.009021 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53eba135-d440-4812-976f-453c8cf4ee4b None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:59:25.034772 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:59:25.043335 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12c460c-7680-42d4-94b6-43247e6a79c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:59:25.044053 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12c460c-7680-42d4-94b6-43247e6a79c3 None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:59:25.044578 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12c460c-7680-42d4-94b6-43247e6a79c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:59:25.044941 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12c460c-7680-42d4-94b6-43247e6a79c3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:59:25.045759 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12c460c-7680-42d4-94b6-43247e6a79c3 None admin] RBAC: Adding query filter params (role.id=8215583bb4224c75b093ae8e3993eb9f, scope.project.id=545d6fe0ae914bcb891a927c746bd18f, user.id=4c68ca93b40941abb187e85c18e2f8fb) {{(pid=80540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 11:59:25.046070 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12c460c-7680-42d4-94b6-43247e6a79c3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:59:25.052487 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12c460c-7680-42d4-94b6-43247e6a79c3 None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:59:25.052955 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 565/1117] 10.222.0.23 () {56 vars in 1374 bytes} [Thu May 19 11:59:24 2022] GET /identity/v3/role_assignments?user.id=4c68ca93b40941abb187e85c18e2f8fb&scope.project.id=545d6fe0ae914bcb891a927c746bd18f&role.id=8215583bb4224c75b093ae8e3993eb9f => generated 259 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:59:27.264696 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b3b8a4-1100-4449-8c02-82bc54bfabe1 None None] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:59:27.265215 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b3b8a4-1100-4449-8c02-82bc54bfabe1 None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:59:27.265215 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b3b8a4-1100-4449-8c02-82bc54bfabe1 None None] PATH_INFO: `/` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:59:27.266532 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 553/1118] 10.222.0.23 () {54 vars in 926 bytes} [Thu May 19 11:59:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 11:59:27.271859 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7f6594-f312-4be1-806d-3461b171d86e None None] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:59:27.272228 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7f6594-f312-4be1-806d-3461b171d86e None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:59:27.272423 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7f6594-f312-4be1-806d-3461b171d86e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:59:27.289618 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.auth.core [None req-bc7f6594-f312-4be1-806d-3461b171d86e None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 11:59:27.290813 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-bc7f6594-f312-4be1-806d-3461b171d86e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 11:59:27.355852 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 566/1119] 10.222.0.23 () {58 vars in 1023 bytes} [Thu May 19 11:59:27 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 11:59:27.379498 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b3b8a4-1100-4449-8c02-82bc54bfabe1 None None] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:59:27.440547 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:59:27.478473 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b3b8a4-1100-4449-8c02-82bc54bfabe1 None None] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:59:27.513186 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10884569-e1b8-476e-ae51-a6489f2bd417 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:59:27.514034 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10884569-e1b8-476e-ae51-a6489f2bd417 None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:59:27.514320 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10884569-e1b8-476e-ae51-a6489f2bd417 None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:59:27.514600 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10884569-e1b8-476e-ae51-a6489f2bd417 None admin] PATH_INFO: `/v3/roles/8215583bb4224c75b093ae8e3993eb9f` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:59:27.516148 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10884569-e1b8-476e-ae51-a6489f2bd417 None admin] RBAC: Authorizing `identity:get_role(role_id=8215583bb4224c75b093ae8e3993eb9f)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:59:27.517475 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10884569-e1b8-476e-ae51-a6489f2bd417 None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:59:27.518846 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 554/1120] 10.222.0.23 () {56 vars in 1177 bytes} [Thu May 19 11:59:27 2022] GET /identity/v3/roles/8215583bb4224c75b093ae8e3993eb9f => generated 223 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:59:27.524687 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7f6594-f312-4be1-806d-3461b171d86e None None] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:59:27.622520 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7f6594-f312-4be1-806d-3461b171d86e None None] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:59:27.641553 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:59:27.655452 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af26b19d-90f9-45d5-aa01-2ca63a78c3e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:59:27.656357 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af26b19d-90f9-45d5-aa01-2ca63a78c3e3 None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:59:27.656640 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af26b19d-90f9-45d5-aa01-2ca63a78c3e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:59:27.656915 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af26b19d-90f9-45d5-aa01-2ca63a78c3e3 None admin] PATH_INFO: `/v3/users/4c68ca93b40941abb187e85c18e2f8fb` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:59:27.658200 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af26b19d-90f9-45d5-aa01-2ca63a78c3e3 None admin] RBAC: Authorizing `identity:get_user(user_id=4c68ca93b40941abb187e85c18e2f8fb)` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:59:27.659651 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af26b19d-90f9-45d5-aa01-2ca63a78c3e3 None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:59:27.661293 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 567/1121] 10.222.0.23 () {56 vars in 1177 bytes} [Thu May 19 11:59:27 2022] GET /identity/v3/users/4c68ca93b40941abb187e85c18e2f8fb => generated 286 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:59:27.667168 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10884569-e1b8-476e-ae51-a6489f2bd417 None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:59:27.764582 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10884569-e1b8-476e-ae51-a6489f2bd417 None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:59:27.792240 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:59:27.798375 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5bc91c-62ea-49be-973b-6640de031445 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:59:27.799236 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5bc91c-62ea-49be-973b-6640de031445 None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:59:27.799516 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5bc91c-62ea-49be-973b-6640de031445 None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:59:27.799808 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5bc91c-62ea-49be-973b-6640de031445 None admin] PATH_INFO: `/v3/projects/545d6fe0ae914bcb891a927c746bd18f` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:59:27.803757 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5bc91c-62ea-49be-973b-6640de031445 None admin] RBAC: Authorizing `identity:get_project(project_id=545d6fe0ae914bcb891a927c746bd18f)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:59:27.803757 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5bc91c-62ea-49be-973b-6640de031445 None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:59:27.804115 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 555/1122] 10.222.0.23 () {56 vars in 1186 bytes} [Thu May 19 11:59:27 2022] GET /identity/v3/projects/545d6fe0ae914bcb891a927c746bd18f => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:59:27.810312 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af26b19d-90f9-45d5-aa01-2ca63a78c3e3 None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:59:27.910014 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af26b19d-90f9-45d5-aa01-2ca63a78c3e3 None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:59:27.944420 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088d8c25-2826-49bc-8abb-dc9fe33f1f83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:59:27.945319 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088d8c25-2826-49bc-8abb-dc9fe33f1f83 None admin] REQUEST_METHOD: `PUT` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:59:27.945607 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088d8c25-2826-49bc-8abb-dc9fe33f1f83 None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:59:27.945889 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088d8c25-2826-49bc-8abb-dc9fe33f1f83 None admin] PATH_INFO: `/v3/projects/545d6fe0ae914bcb891a927c746bd18f/users/4c68ca93b40941abb187e85c18e2f8fb/roles/8215583bb4224c75b093ae8e3993eb9f` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:59:27.948376 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088d8c25-2826-49bc-8abb-dc9fe33f1f83 None admin] RBAC: Authorizing `identity:create_grant(project_id=545d6fe0ae914bcb891a927c746bd18f, user_id=4c68ca93b40941abb187e85c18e2f8fb, role_id=8215583bb4224c75b093ae8e3993eb9f)` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:59:27.949839 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088d8c25-2826-49bc-8abb-dc9fe33f1f83 None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:59:28.021919 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 568/1123] 10.222.0.23 () {58 vars in 1439 bytes} [Thu May 19 11:59:27 2022] PUT /identity/v3/projects/545d6fe0ae914bcb891a927c746bd18f/users/4c68ca93b40941abb187e85c18e2f8fb/roles/8215583bb4224c75b093ae8e3993eb9f => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 11:59:30.215008 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09034b9-2105-462c-9d4e-dba584b6f472 None None] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:59:30.215509 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09034b9-2105-462c-9d4e-dba584b6f472 None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:59:30.215509 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09034b9-2105-462c-9d4e-dba584b6f472 None None] PATH_INFO: `/` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:59:30.217308 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 556/1124] 10.222.0.23 () {54 vars in 926 bytes} [Thu May 19 11:59:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 11:59:30.222544 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c06bb8-22d6-4852-b09d-c1192b8729a9 None None] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:59:30.222946 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c06bb8-22d6-4852-b09d-c1192b8729a9 None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:59:30.223115 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c06bb8-22d6-4852-b09d-c1192b8729a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:59:30.240635 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.auth.core [None req-c5c06bb8-22d6-4852-b09d-c1192b8729a9 None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 11:59:30.241838 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-c5c06bb8-22d6-4852-b09d-c1192b8729a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 11:59:30.287625 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:59:30.312070 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 569/1125] 10.222.0.23 () {58 vars in 1023 bytes} [Thu May 19 11:59:30 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 11:59:30.336466 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09034b9-2105-462c-9d4e-dba584b6f472 None None] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:59:30.442526 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09034b9-2105-462c-9d4e-dba584b6f472 None None] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:59:30.476023 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e878f22f-d649-4dd3-a511-11b1f2af377e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:59:30.476810 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e878f22f-d649-4dd3-a511-11b1f2af377e None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:59:30.477095 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e878f22f-d649-4dd3-a511-11b1f2af377e None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:59:30.477370 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e878f22f-d649-4dd3-a511-11b1f2af377e None admin] PATH_INFO: `/v3/roles/8215583bb4224c75b093ae8e3993eb9f` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:59:30.478840 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e878f22f-d649-4dd3-a511-11b1f2af377e None admin] RBAC: Authorizing `identity:get_role(role_id=8215583bb4224c75b093ae8e3993eb9f)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:59:30.480247 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e878f22f-d649-4dd3-a511-11b1f2af377e None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:59:30.482125 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 557/1126] 10.222.0.23 () {56 vars in 1177 bytes} [Thu May 19 11:59:30 2022] GET /identity/v3/roles/8215583bb4224c75b093ae8e3993eb9f => generated 223 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:59:30.488286 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c06bb8-22d6-4852-b09d-c1192b8729a9 None None] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:59:30.586938 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c06bb8-22d6-4852-b09d-c1192b8729a9 None None] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:59:30.625736 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c593d8-150a-4a7b-b8c8-ea1c4159f80e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:59:30.626137 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c593d8-150a-4a7b-b8c8-ea1c4159f80e None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:59:30.626424 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c593d8-150a-4a7b-b8c8-ea1c4159f80e None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:59:30.626704 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c593d8-150a-4a7b-b8c8-ea1c4159f80e None admin] PATH_INFO: `/v3/users/4c68ca93b40941abb187e85c18e2f8fb` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:59:30.627933 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c593d8-150a-4a7b-b8c8-ea1c4159f80e None admin] RBAC: Authorizing `identity:get_user(user_id=4c68ca93b40941abb187e85c18e2f8fb)` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:59:30.629398 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c593d8-150a-4a7b-b8c8-ea1c4159f80e None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:59:30.631048 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 570/1127] 10.222.0.23 () {56 vars in 1177 bytes} [Thu May 19 11:59:30 2022] GET /identity/v3/users/4c68ca93b40941abb187e85c18e2f8fb => generated 286 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:59:30.636821 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e878f22f-d649-4dd3-a511-11b1f2af377e None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:59:30.700339 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:59:30.736979 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e878f22f-d649-4dd3-a511-11b1f2af377e None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:59:30.770724 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c60340-c5b2-486b-8556-a4ebe9b9e95b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:59:30.771370 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c60340-c5b2-486b-8556-a4ebe9b9e95b None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:59:30.771656 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c60340-c5b2-486b-8556-a4ebe9b9e95b None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:59:30.771962 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c60340-c5b2-486b-8556-a4ebe9b9e95b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:59:30.838956 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c60340-c5b2-486b-8556-a4ebe9b9e95b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:59:30.840427 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c60340-c5b2-486b-8556-a4ebe9b9e95b None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:59:30.901963 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:59:30.938613 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 558/1128] 10.222.0.23 () {58 vars in 1282 bytes} [Thu May 19 11:59:30 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 19 11:59:30.944593 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c593d8-150a-4a7b-b8c8-ea1c4159f80e None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:59:30.994026 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:59:31.048939 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c593d8-150a-4a7b-b8c8-ea1c4159f80e None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:59:31.083916 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbb5aaf-1059-4e8e-beb9-3a4dea8b36aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:59:31.084777 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbb5aaf-1059-4e8e-beb9-3a4dea8b36aa None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:59:31.085061 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbb5aaf-1059-4e8e-beb9-3a4dea8b36aa None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:59:31.085339 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbb5aaf-1059-4e8e-beb9-3a4dea8b36aa None admin] PATH_INFO: `/v3/projects/545d6fe0ae914bcb891a927c746bd18f` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:59:31.086545 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bbb5aaf-1059-4e8e-beb9-3a4dea8b36aa None admin] RBAC: Authorizing `identity:get_project(project_id=545d6fe0ae914bcb891a927c746bd18f)` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:59:31.088028 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bbb5aaf-1059-4e8e-beb9-3a4dea8b36aa None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:59:31.089727 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 571/1129] 10.222.0.23 () {56 vars in 1186 bytes} [Thu May 19 11:59:30 2022] GET /identity/v3/projects/545d6fe0ae914bcb891a927c746bd18f => generated 311 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:59:31.096524 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c60340-c5b2-486b-8556-a4ebe9b9e95b None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:59:31.193952 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c60340-c5b2-486b-8556-a4ebe9b9e95b None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:59:31.228421 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02666c72-eb98-4199-998e-c14e7736ba41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:59:31.229069 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02666c72-eb98-4199-998e-c14e7736ba41 None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:59:31.229351 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02666c72-eb98-4199-998e-c14e7736ba41 None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:59:31.229627 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02666c72-eb98-4199-998e-c14e7736ba41 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:59:31.230455 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02666c72-eb98-4199-998e-c14e7736ba41 None admin] RBAC: Adding query filter params (role.id=8215583bb4224c75b093ae8e3993eb9f, scope.project.id=545d6fe0ae914bcb891a927c746bd18f, user.id=4c68ca93b40941abb187e85c18e2f8fb) {{(pid=80539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 11:59:31.230774 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02666c72-eb98-4199-998e-c14e7736ba41 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:59:31.232447 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02666c72-eb98-4199-998e-c14e7736ba41 None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:59:31.238281 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 559/1130] 10.222.0.23 () {56 vars in 1374 bytes} [Thu May 19 11:59:31 2022] GET /identity/v3/role_assignments?user.id=4c68ca93b40941abb187e85c18e2f8fb&scope.project.id=545d6fe0ae914bcb891a927c746bd18f&role.id=8215583bb4224c75b093ae8e3993eb9f => generated 608 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:59:33.480932 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc6de8d-6935-405d-a4db-5ed9c790df5b None None] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:59:33.481491 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc6de8d-6935-405d-a4db-5ed9c790df5b None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:59:33.481491 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc6de8d-6935-405d-a4db-5ed9c790df5b None None] PATH_INFO: `/` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:59:33.483227 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 572/1131] 10.222.0.23 () {54 vars in 926 bytes} [Thu May 19 11:59:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 11:59:33.488577 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e84e2d-07b6-4001-9bd6-03bf6c1d4deb None None] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:59:33.489018 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e84e2d-07b6-4001-9bd6-03bf6c1d4deb None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:59:33.489137 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e84e2d-07b6-4001-9bd6-03bf6c1d4deb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:59:33.505919 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.auth.core [None req-a7e84e2d-07b6-4001-9bd6-03bf6c1d4deb None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 11:59:33.507109 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-a7e84e2d-07b6-4001-9bd6-03bf6c1d4deb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 11:59:33.531068 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:59:33.574792 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 560/1132] 10.222.0.23 () {58 vars in 1023 bytes} [Thu May 19 11:59:33 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 11:59:33.598473 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc6de8d-6935-405d-a4db-5ed9c790df5b None None] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:59:33.702748 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc6de8d-6935-405d-a4db-5ed9c790df5b None None] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:59:33.716866 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:59:33.737250 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0a4d3a-9786-4b7e-9aa5-0b74d1056896 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:59:33.738007 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0a4d3a-9786-4b7e-9aa5-0b74d1056896 None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:59:33.738291 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0a4d3a-9786-4b7e-9aa5-0b74d1056896 None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:59:33.738567 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0a4d3a-9786-4b7e-9aa5-0b74d1056896 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:59:33.739751 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0a4d3a-9786-4b7e-9aa5-0b74d1056896 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:59:33.741219 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0a4d3a-9786-4b7e-9aa5-0b74d1056896 None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:59:33.743095 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 573/1133] 10.222.0.23 () {56 vars in 1108 bytes} [Thu May 19 11:59:33 2022] GET /identity/v3/domains/default => generated 199 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:59:33.749994 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e84e2d-07b6-4001-9bd6-03bf6c1d4deb None None] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:59:33.846219 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e84e2d-07b6-4001-9bd6-03bf6c1d4deb None None] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:59:33.879645 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2f8bc9-f913-44b3-9133-2a8d219d5dd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:59:33.880330 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2f8bc9-f913-44b3-9133-2a8d219d5dd4 None admin] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:59:33.880617 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2f8bc9-f913-44b3-9133-2a8d219d5dd4 None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:59:33.880898 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2f8bc9-f913-44b3-9133-2a8d219d5dd4 None admin] PATH_INFO: `/v3/projects` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:59:33.881618 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b2f8bc9-f913-44b3-9133-2a8d219d5dd4 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:59:33.883054 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b2f8bc9-f913-44b3-9133-2a8d219d5dd4 None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:59:33.992851 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 561/1134] 10.222.0.23 () {60 vars in 1141 bytes} [Thu May 19 11:59:33 2022] POST /identity/v3/projects => generated 311 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 11:59:36.226025 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3cc81a-75a5-4d02-9f05-4fd16f0537c1 None None] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:59:36.226591 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3cc81a-75a5-4d02-9f05-4fd16f0537c1 None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:59:36.226591 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3cc81a-75a5-4d02-9f05-4fd16f0537c1 None None] PATH_INFO: `/` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:59:36.227947 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 574/1135] 10.222.0.23 () {54 vars in 926 bytes} [Thu May 19 11:59:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 11:59:36.233334 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db66f252-39e9-4635-8ee2-379efacaf770 None None] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:59:36.233767 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db66f252-39e9-4635-8ee2-379efacaf770 None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:59:36.233898 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db66f252-39e9-4635-8ee2-379efacaf770 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:59:36.250939 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.auth.core [None req-db66f252-39e9-4635-8ee2-379efacaf770 None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 11:59:36.252168 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-db66f252-39e9-4635-8ee2-379efacaf770 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 11:59:36.259306 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:59:36.316837 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 562/1136] 10.222.0.23 () {58 vars in 1023 bytes} [Thu May 19 11:59:36 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 11:59:36.342145 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3cc81a-75a5-4d02-9f05-4fd16f0537c1 None None] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:59:36.445027 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3cc81a-75a5-4d02-9f05-4fd16f0537c1 None None] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:59:36.478651 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03338ec1-5326-4a2a-b670-9469fa8c54aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:59:36.479369 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03338ec1-5326-4a2a-b670-9469fa8c54aa None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:59:36.479651 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03338ec1-5326-4a2a-b670-9469fa8c54aa None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:59:36.479957 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03338ec1-5326-4a2a-b670-9469fa8c54aa None admin] PATH_INFO: `/v3/domains/default` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:59:36.481106 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03338ec1-5326-4a2a-b670-9469fa8c54aa None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:59:36.482792 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03338ec1-5326-4a2a-b670-9469fa8c54aa None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:59:36.484516 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 575/1137] 10.222.0.23 () {56 vars in 1108 bytes} [Thu May 19 11:59:36 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:59:36.490464 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db66f252-39e9-4635-8ee2-379efacaf770 None None] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:59:36.588824 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db66f252-39e9-4635-8ee2-379efacaf770 None None] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:59:36.621586 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112136ab-d87d-4e35-86b5-256c2eb1e34a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:59:36.622863 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112136ab-d87d-4e35-86b5-256c2eb1e34a None admin] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:59:36.623146 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112136ab-d87d-4e35-86b5-256c2eb1e34a None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:59:36.623500 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112136ab-d87d-4e35-86b5-256c2eb1e34a None admin] PATH_INFO: `/v3/users` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:59:36.624251 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112136ab-d87d-4e35-86b5-256c2eb1e34a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:59:36.625676 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112136ab-d87d-4e35-86b5-256c2eb1e34a None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:59:36.685761 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:59:36.691492 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 563/1138] 10.222.0.23 () {60 vars in 1132 bytes} [Thu May 19 11:59:36 2022] POST /identity/v3/users => generated 286 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 11:59:38.984541 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad5d60d-8009-4cc8-a71f-0446fc93d874 None None] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:59:38.985047 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad5d60d-8009-4cc8-a71f-0446fc93d874 None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:59:38.985162 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad5d60d-8009-4cc8-a71f-0446fc93d874 None None] PATH_INFO: `/` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:59:38.986483 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 576/1139] 10.222.0.23 () {54 vars in 926 bytes} [Thu May 19 11:59:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 11:59:38.991939 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c4f314-d38f-4c77-b3c8-3872583f8204 None None] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:59:38.992363 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c4f314-d38f-4c77-b3c8-3872583f8204 None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:59:38.992499 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c4f314-d38f-4c77-b3c8-3872583f8204 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:59:39.009086 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.auth.core [None req-75c4f314-d38f-4c77-b3c8-3872583f8204 None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 11:59:39.010283 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-75c4f314-d38f-4c77-b3c8-3872583f8204 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 11:59:39.083972 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 564/1140] 10.222.0.23 () {58 vars in 1023 bytes} [Thu May 19 11:59:38 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 11:59:39.107957 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad5d60d-8009-4cc8-a71f-0446fc93d874 None None] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:59:39.159007 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:59:39.207486 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad5d60d-8009-4cc8-a71f-0446fc93d874 None None] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:59:39.240241 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d45859c-76c7-4857-94a3-ca069c488a59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:59:39.241018 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d45859c-76c7-4857-94a3-ca069c488a59 None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:59:39.241300 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d45859c-76c7-4857-94a3-ca069c488a59 None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:59:39.241573 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d45859c-76c7-4857-94a3-ca069c488a59 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:59:39.251902 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d45859c-76c7-4857-94a3-ca069c488a59 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 11:59:39.259309 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application [None req-9d45859c-76c7-4857-94a3-ca069c488a59 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 11:59:39.259309 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 11:59:39.259309 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 11:59:39.259309 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 11:59:39.259309 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 11:59:39.259309 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 11:59:39.259309 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 11:59:39.259309 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 11:59:39.259309 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 11:59:39.259309 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 11:59:39.259309 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 11:59:39.259309 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 11:59:39.259309 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 11:59:39.259309 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 11:59:39.259309 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 11:59:39.259309 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 11:59:39.260503 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 11:59:39.260503 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 11:59:39.260503 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 11:59:39.260503 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 11:59:39.260503 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 11:59:39.260503 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 11:59:39.260503 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 11:59:39.260503 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 11:59:39.260503 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 11:59:39.260503 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.get_or_create( May 19 11:59:39.260503 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 11:59:39.260503 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application with Lock( May 19 11:59:39.260503 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 11:59:39.260503 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._enter() May 19 11:59:39.260503 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 11:59:39.260503 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 11:59:39.261774 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 11:59:39.261774 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.creator() May 19 11:59:39.261774 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 11:59:39.261774 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application created_value = creator( May 19 11:59:39.261774 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 11:59:39.261774 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 11:59:39.261774 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 11:59:39.261774 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 11:59:39.261774 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 11:59:39.261774 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 11:59:39.261774 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 11:59:39.261774 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application May 19 11:59:39.261774 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 577/1141] 10.222.0.23 () {56 vars in 1096 bytes} [Thu May 19 11:59:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 11:59:39.267102 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c4f314-d38f-4c77-b3c8-3872583f8204 None None] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:59:39.375228 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c4f314-d38f-4c77-b3c8-3872583f8204 None None] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:59:39.404342 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:59:39.412099 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd66b36e-546f-43fa-a157-4eb6256b7f17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:59:39.415114 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd66b36e-546f-43fa-a157-4eb6256b7f17 None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:59:39.415114 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd66b36e-546f-43fa-a157-4eb6256b7f17 None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:59:39.415114 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd66b36e-546f-43fa-a157-4eb6256b7f17 None admin] PATH_INFO: `/v3/roles` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:59:39.415114 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd66b36e-546f-43fa-a157-4eb6256b7f17 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 11:59:39.415114 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd66b36e-546f-43fa-a157-4eb6256b7f17 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:59:39.415770 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd66b36e-546f-43fa-a157-4eb6256b7f17 None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:59:39.424040 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 565/1142] 10.222.0.23 () {56 vars in 1099 bytes} [Thu May 19 11:59:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:59:39.429689 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d45859c-76c7-4857-94a3-ca069c488a59 None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:59:39.529820 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d45859c-76c7-4857-94a3-ca069c488a59 None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:59:39.539957 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:59:39.563197 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d88e42-811f-4eb9-9409-60b9be84458c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:59:39.564186 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d88e42-811f-4eb9-9409-60b9be84458c None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:59:39.564479 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d88e42-811f-4eb9-9409-60b9be84458c None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:59:39.564760 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d88e42-811f-4eb9-9409-60b9be84458c None admin] PATH_INFO: `/v3/users/0c62c5e7243b44488bb93d7b8dfef9b4` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:59:39.579110 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d88e42-811f-4eb9-9409-60b9be84458c None admin] RBAC: Authorizing `identity:get_user(user_id=0c62c5e7243b44488bb93d7b8dfef9b4)` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:59:39.580609 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d88e42-811f-4eb9-9409-60b9be84458c None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:59:39.582277 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 578/1143] 10.222.0.23 () {56 vars in 1177 bytes} [Thu May 19 11:59:39 2022] GET /identity/v3/users/0c62c5e7243b44488bb93d7b8dfef9b4 => generated 286 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:59:39.588045 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd66b36e-546f-43fa-a157-4eb6256b7f17 None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:59:39.688228 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd66b36e-546f-43fa-a157-4eb6256b7f17 None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:59:39.727572 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff35fede-e618-4c98-a258-fda9250fbe46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:59:39.728247 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff35fede-e618-4c98-a258-fda9250fbe46 None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:59:39.728534 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff35fede-e618-4c98-a258-fda9250fbe46 None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:59:39.728812 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff35fede-e618-4c98-a258-fda9250fbe46 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:59:39.785724 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:59:39.795539 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff35fede-e618-4c98-a258-fda9250fbe46 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:59:39.803930 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff35fede-e618-4c98-a258-fda9250fbe46 None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:59:39.893555 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 566/1144] 10.222.0.23 () {58 vars in 1282 bytes} [Thu May 19 11:59:39 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 11:59:39.899465 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d88e42-811f-4eb9-9409-60b9be84458c None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:59:40.001272 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d88e42-811f-4eb9-9409-60b9be84458c None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:59:40.035449 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30bb9325-07c6-4894-a0b5-b5749ce94214 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:59:40.036452 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bb9325-07c6-4894-a0b5-b5749ce94214 None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:59:40.036688 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bb9325-07c6-4894-a0b5-b5749ce94214 None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:59:40.036969 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bb9325-07c6-4894-a0b5-b5749ce94214 None admin] PATH_INFO: `/v3/projects/f8652d1eff8d468eb8adeb5df6062ee5` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:59:40.038527 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30bb9325-07c6-4894-a0b5-b5749ce94214 None admin] RBAC: Authorizing `identity:get_project(project_id=f8652d1eff8d468eb8adeb5df6062ee5)` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:59:40.040012 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30bb9325-07c6-4894-a0b5-b5749ce94214 None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:59:40.041743 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 579/1145] 10.222.0.23 () {56 vars in 1186 bytes} [Thu May 19 11:59:39 2022] GET /identity/v3/projects/f8652d1eff8d468eb8adeb5df6062ee5 => generated 311 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:59:40.048939 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff35fede-e618-4c98-a258-fda9250fbe46 None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:59:40.136885 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:59:40.146580 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff35fede-e618-4c98-a258-fda9250fbe46 None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:59:40.178843 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3c6644-37d5-4027-aea4-3c5cfbc137f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:59:40.179483 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3c6644-37d5-4027-aea4-3c5cfbc137f6 None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:59:40.179765 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3c6644-37d5-4027-aea4-3c5cfbc137f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:59:40.180072 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3c6644-37d5-4027-aea4-3c5cfbc137f6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:59:40.180892 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3c6644-37d5-4027-aea4-3c5cfbc137f6 None admin] RBAC: Adding query filter params (role.id=9db5671c65d3414ea2682afa2ccf1403, scope.project.id=f8652d1eff8d468eb8adeb5df6062ee5, user.id=0c62c5e7243b44488bb93d7b8dfef9b4) {{(pid=80539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 11:59:40.181202 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3c6644-37d5-4027-aea4-3c5cfbc137f6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:59:40.182619 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3c6644-37d5-4027-aea4-3c5cfbc137f6 None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:59:40.188423 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 567/1146] 10.222.0.23 () {56 vars in 1374 bytes} [Thu May 19 11:59:40 2022] GET /identity/v3/role_assignments?user.id=0c62c5e7243b44488bb93d7b8dfef9b4&scope.project.id=f8652d1eff8d468eb8adeb5df6062ee5&role.id=9db5671c65d3414ea2682afa2ccf1403 => generated 259 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:59:42.373898 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a658154-3090-466c-8031-a162a0bfbc09 None None] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:59:42.374471 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a658154-3090-466c-8031-a162a0bfbc09 None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:59:42.374471 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a658154-3090-466c-8031-a162a0bfbc09 None None] PATH_INFO: `/` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:59:42.375885 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 580/1147] 10.222.0.23 () {54 vars in 926 bytes} [Thu May 19 11:59:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 11:59:42.381593 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898eecef-55f5-4f10-883f-cdfdb49a31ed None None] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:59:42.382079 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898eecef-55f5-4f10-883f-cdfdb49a31ed None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:59:42.382175 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898eecef-55f5-4f10-883f-cdfdb49a31ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:59:42.398643 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.auth.core [None req-898eecef-55f5-4f10-883f-cdfdb49a31ed None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 11:59:42.399867 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-898eecef-55f5-4f10-883f-cdfdb49a31ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 11:59:42.464655 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 568/1148] 10.222.0.23 () {58 vars in 1023 bytes} [Thu May 19 11:59:42 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 11:59:42.488220 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a658154-3090-466c-8031-a162a0bfbc09 None None] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:59:42.523256 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:59:42.588905 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a658154-3090-466c-8031-a162a0bfbc09 None None] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:59:42.623976 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de5457a-0609-4839-998d-b1e89fabb1c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:59:42.624697 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de5457a-0609-4839-998d-b1e89fabb1c8 None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:59:42.624986 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de5457a-0609-4839-998d-b1e89fabb1c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:59:42.625266 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de5457a-0609-4839-998d-b1e89fabb1c8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:59:42.635324 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de5457a-0609-4839-998d-b1e89fabb1c8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 11:59:42.643817 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application [None req-1de5457a-0609-4839-998d-b1e89fabb1c8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 11:59:42.643817 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 11:59:42.643817 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 11:59:42.643817 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 11:59:42.643817 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 11:59:42.643817 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 11:59:42.643817 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 11:59:42.643817 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 11:59:42.643817 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 11:59:42.643817 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 11:59:42.643817 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 11:59:42.643817 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 11:59:42.643817 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 11:59:42.643817 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 11:59:42.643817 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 11:59:42.643817 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 11:59:42.644975 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 11:59:42.644975 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 11:59:42.644975 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 11:59:42.644975 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 11:59:42.644975 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 11:59:42.644975 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 11:59:42.644975 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 11:59:42.644975 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 11:59:42.644975 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 11:59:42.644975 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.get_or_create( May 19 11:59:42.644975 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 11:59:42.644975 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application with Lock( May 19 11:59:42.644975 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 11:59:42.644975 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._enter() May 19 11:59:42.644975 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 11:59:42.644975 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 11:59:42.646035 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 11:59:42.646035 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.creator() May 19 11:59:42.646035 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 11:59:42.646035 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application created_value = creator( May 19 11:59:42.646035 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 11:59:42.646035 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 11:59:42.646035 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 11:59:42.646035 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 11:59:42.646035 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 11:59:42.646035 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 11:59:42.646035 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 11:59:42.646035 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application May 19 11:59:42.646035 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 581/1149] 10.222.0.23 () {56 vars in 1096 bytes} [Thu May 19 11:59:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 11:59:42.651202 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898eecef-55f5-4f10-883f-cdfdb49a31ed None None] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:59:42.734042 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:59:42.749338 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898eecef-55f5-4f10-883f-cdfdb49a31ed None None] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:59:42.783347 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ef23a5-7c30-4da2-a484-b7d2b01e93da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:59:42.784046 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ef23a5-7c30-4da2-a484-b7d2b01e93da None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:59:42.784334 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ef23a5-7c30-4da2-a484-b7d2b01e93da None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:59:42.784611 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ef23a5-7c30-4da2-a484-b7d2b01e93da None admin] PATH_INFO: `/v3/roles` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:59:42.785364 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ef23a5-7c30-4da2-a484-b7d2b01e93da None admin] RBAC: Adding query filter params (name=admin) {{(pid=80539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 11:59:42.785675 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ef23a5-7c30-4da2-a484-b7d2b01e93da None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:59:42.787055 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ef23a5-7c30-4da2-a484-b7d2b01e93da None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:59:42.795129 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 569/1150] 10.222.0.23 () {56 vars in 1099 bytes} [Thu May 19 11:59:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:59:42.801006 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de5457a-0609-4839-998d-b1e89fabb1c8 None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:59:42.894599 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:59:42.900498 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de5457a-0609-4839-998d-b1e89fabb1c8 None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:59:42.934585 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8ae598-2bd5-4fe7-b010-b36c7d9c43f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:59:42.935642 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8ae598-2bd5-4fe7-b010-b36c7d9c43f2 None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:59:42.935975 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8ae598-2bd5-4fe7-b010-b36c7d9c43f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:59:42.936258 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8ae598-2bd5-4fe7-b010-b36c7d9c43f2 None admin] PATH_INFO: `/v3/users/0c62c5e7243b44488bb93d7b8dfef9b4` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:59:42.937459 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8ae598-2bd5-4fe7-b010-b36c7d9c43f2 None admin] RBAC: Authorizing `identity:get_user(user_id=0c62c5e7243b44488bb93d7b8dfef9b4)` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:59:42.938917 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8ae598-2bd5-4fe7-b010-b36c7d9c43f2 None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:59:42.940585 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 582/1151] 10.222.0.23 () {56 vars in 1177 bytes} [Thu May 19 11:59:42 2022] GET /identity/v3/users/0c62c5e7243b44488bb93d7b8dfef9b4 => generated 286 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:59:42.946251 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ef23a5-7c30-4da2-a484-b7d2b01e93da None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:59:43.045837 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ef23a5-7c30-4da2-a484-b7d2b01e93da None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:59:43.079742 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df23142-11a5-4db8-847c-c2b9988db4fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:59:43.080631 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df23142-11a5-4db8-847c-c2b9988db4fd None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:59:43.080916 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df23142-11a5-4db8-847c-c2b9988db4fd None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:59:43.081193 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df23142-11a5-4db8-847c-c2b9988db4fd None admin] PATH_INFO: `/v3/projects/f8652d1eff8d468eb8adeb5df6062ee5` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:59:43.082375 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df23142-11a5-4db8-847c-c2b9988db4fd None admin] RBAC: Authorizing `identity:get_project(project_id=f8652d1eff8d468eb8adeb5df6062ee5)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:59:43.083837 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df23142-11a5-4db8-847c-c2b9988db4fd None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:59:43.085531 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 570/1152] 10.222.0.23 () {56 vars in 1186 bytes} [Thu May 19 11:59:42 2022] GET /identity/v3/projects/f8652d1eff8d468eb8adeb5df6062ee5 => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:59:43.091975 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8ae598-2bd5-4fe7-b010-b36c7d9c43f2 None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:59:43.197316 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8ae598-2bd5-4fe7-b010-b36c7d9c43f2 None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:59:43.231139 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22abbd9-919e-4264-90af-28a4c41c6f7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:59:43.232256 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22abbd9-919e-4264-90af-28a4c41c6f7d None admin] REQUEST_METHOD: `PUT` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:59:43.232642 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22abbd9-919e-4264-90af-28a4c41c6f7d None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:59:43.232938 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22abbd9-919e-4264-90af-28a4c41c6f7d None admin] PATH_INFO: `/v3/projects/f8652d1eff8d468eb8adeb5df6062ee5/users/0c62c5e7243b44488bb93d7b8dfef9b4/roles/9db5671c65d3414ea2682afa2ccf1403` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:59:43.234878 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22abbd9-919e-4264-90af-28a4c41c6f7d None admin] RBAC: Authorizing `identity:create_grant(project_id=f8652d1eff8d468eb8adeb5df6062ee5, user_id=0c62c5e7243b44488bb93d7b8dfef9b4, role_id=9db5671c65d3414ea2682afa2ccf1403)` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:59:43.236434 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22abbd9-919e-4264-90af-28a4c41c6f7d None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:59:43.342033 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 583/1153] 10.222.0.23 () {58 vars in 1439 bytes} [Thu May 19 11:59:43 2022] PUT /identity/v3/projects/f8652d1eff8d468eb8adeb5df6062ee5/users/0c62c5e7243b44488bb93d7b8dfef9b4/roles/9db5671c65d3414ea2682afa2ccf1403 => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 11:59:45.561714 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b556e13-c717-4578-a785-d23422701b5c None None] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:59:45.562246 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b556e13-c717-4578-a785-d23422701b5c None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:59:45.562246 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b556e13-c717-4578-a785-d23422701b5c None None] PATH_INFO: `/` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:59:45.563707 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 571/1154] 10.222.0.23 () {54 vars in 926 bytes} [Thu May 19 11:59:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 11:59:45.569004 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68009dc0-00d5-4df4-bce3-647aa5c4fc2f None None] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:59:45.569422 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68009dc0-00d5-4df4-bce3-647aa5c4fc2f None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:59:45.569578 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68009dc0-00d5-4df4-bce3-647aa5c4fc2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:59:45.586516 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.auth.core [None req-68009dc0-00d5-4df4-bce3-647aa5c4fc2f None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 11:59:45.587717 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-68009dc0-00d5-4df4-bce3-647aa5c4fc2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 11:59:45.605548 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:59:45.656763 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 584/1155] 10.222.0.23 () {58 vars in 1023 bytes} [Thu May 19 11:59:45 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 11:59:45.680421 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b556e13-c717-4578-a785-d23422701b5c None None] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:59:45.696587 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:59:45.793982 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b556e13-c717-4578-a785-d23422701b5c None None] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:59:45.827837 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901441b8-7fbb-4c64-8788-59d9a16ab456 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:59:45.828557 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901441b8-7fbb-4c64-8788-59d9a16ab456 None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:59:45.828839 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901441b8-7fbb-4c64-8788-59d9a16ab456 None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:59:45.829112 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901441b8-7fbb-4c64-8788-59d9a16ab456 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:59:45.839759 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901441b8-7fbb-4c64-8788-59d9a16ab456 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 11:59:45.848097 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application [None req-901441b8-7fbb-4c64-8788-59d9a16ab456 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 11:59:45.848097 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 11:59:45.848097 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 11:59:45.848097 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 11:59:45.848097 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 11:59:45.848097 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 11:59:45.848097 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 11:59:45.848097 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 11:59:45.848097 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 11:59:45.848097 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 11:59:45.848097 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 11:59:45.848097 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 11:59:45.848097 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 11:59:45.848097 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 11:59:45.848097 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 11:59:45.848097 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 11:59:45.849230 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 11:59:45.849230 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 11:59:45.849230 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 11:59:45.849230 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 11:59:45.849230 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 11:59:45.849230 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 11:59:45.849230 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 11:59:45.849230 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 11:59:45.849230 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 11:59:45.849230 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.get_or_create( May 19 11:59:45.849230 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 11:59:45.849230 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application with Lock( May 19 11:59:45.849230 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 11:59:45.849230 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._enter() May 19 11:59:45.849230 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 11:59:45.849230 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 11:59:45.850180 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 11:59:45.850180 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.creator() May 19 11:59:45.850180 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 11:59:45.850180 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application created_value = creator( May 19 11:59:45.850180 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 11:59:45.850180 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 11:59:45.850180 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 11:59:45.850180 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 11:59:45.850180 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 11:59:45.850180 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 11:59:45.850180 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 11:59:45.850180 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application May 19 11:59:45.850180 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 572/1156] 10.222.0.23 () {56 vars in 1096 bytes} [Thu May 19 11:59:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 11:59:45.855389 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68009dc0-00d5-4df4-bce3-647aa5c4fc2f None None] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:59:45.953116 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68009dc0-00d5-4df4-bce3-647aa5c4fc2f None None] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:59:45.986399 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0db6dd-55cd-45c2-8dbf-840d645d3fa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:59:45.987058 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0db6dd-55cd-45c2-8dbf-840d645d3fa2 None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:59:45.987343 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0db6dd-55cd-45c2-8dbf-840d645d3fa2 None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:59:45.987620 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0db6dd-55cd-45c2-8dbf-840d645d3fa2 None admin] PATH_INFO: `/v3/roles` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:59:45.988394 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0db6dd-55cd-45c2-8dbf-840d645d3fa2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 11:59:45.988704 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0db6dd-55cd-45c2-8dbf-840d645d3fa2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:59:45.990082 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0db6dd-55cd-45c2-8dbf-840d645d3fa2 None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:59:45.998601 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 585/1157] 10.222.0.23 () {56 vars in 1099 bytes} [Thu May 19 11:59:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:59:46.004345 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901441b8-7fbb-4c64-8788-59d9a16ab456 None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:59:46.083394 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:59:46.108579 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901441b8-7fbb-4c64-8788-59d9a16ab456 None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:59:46.142250 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ffc7e3-0a21-4393-b497-71978487be68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:59:46.143109 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ffc7e3-0a21-4393-b497-71978487be68 None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:59:46.143391 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ffc7e3-0a21-4393-b497-71978487be68 None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:59:46.143664 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ffc7e3-0a21-4393-b497-71978487be68 None admin] PATH_INFO: `/v3/users/0c62c5e7243b44488bb93d7b8dfef9b4` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:59:46.144858 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ffc7e3-0a21-4393-b497-71978487be68 None admin] RBAC: Authorizing `identity:get_user(user_id=0c62c5e7243b44488bb93d7b8dfef9b4)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:59:46.146283 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ffc7e3-0a21-4393-b497-71978487be68 None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:59:46.147926 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 573/1158] 10.222.0.23 () {56 vars in 1177 bytes} [Thu May 19 11:59:46 2022] GET /identity/v3/users/0c62c5e7243b44488bb93d7b8dfef9b4 => generated 286 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:59:46.153880 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0db6dd-55cd-45c2-8dbf-840d645d3fa2 None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:59:46.172173 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:59:46.253222 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0db6dd-55cd-45c2-8dbf-840d645d3fa2 None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:59:46.286925 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe9c1db-6e23-4dde-b467-7886f21bf508 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:59:46.287564 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe9c1db-6e23-4dde-b467-7886f21bf508 None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:59:46.287875 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe9c1db-6e23-4dde-b467-7886f21bf508 None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:59:46.288153 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe9c1db-6e23-4dde-b467-7886f21bf508 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:59:46.354178 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe9c1db-6e23-4dde-b467-7886f21bf508 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:59:46.355576 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe9c1db-6e23-4dde-b467-7886f21bf508 None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:59:46.373681 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:59:46.454618 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 586/1159] 10.222.0.23 () {58 vars in 1282 bytes} [Thu May 19 11:59:46 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 11:59:46.460600 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ffc7e3-0a21-4393-b497-71978487be68 None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:59:46.557355 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ffc7e3-0a21-4393-b497-71978487be68 None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:59:46.591896 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27e53ac-2f7e-4ad9-958a-ec38a1168d8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:59:46.592782 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27e53ac-2f7e-4ad9-958a-ec38a1168d8c None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:59:46.593063 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27e53ac-2f7e-4ad9-958a-ec38a1168d8c None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:59:46.593336 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27e53ac-2f7e-4ad9-958a-ec38a1168d8c None admin] PATH_INFO: `/v3/projects/f8652d1eff8d468eb8adeb5df6062ee5` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:59:46.594490 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27e53ac-2f7e-4ad9-958a-ec38a1168d8c None admin] RBAC: Authorizing `identity:get_project(project_id=f8652d1eff8d468eb8adeb5df6062ee5)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:59:46.595945 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27e53ac-2f7e-4ad9-958a-ec38a1168d8c None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:59:46.597609 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 574/1160] 10.222.0.23 () {56 vars in 1186 bytes} [Thu May 19 11:59:46 2022] GET /identity/v3/projects/f8652d1eff8d468eb8adeb5df6062ee5 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:59:46.603841 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe9c1db-6e23-4dde-b467-7886f21bf508 None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:59:46.703886 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe9c1db-6e23-4dde-b467-7886f21bf508 None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:59:46.719867 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:59:46.737571 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c00659-196b-459d-b0c3-75489c4d388f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:59:46.738228 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c00659-196b-459d-b0c3-75489c4d388f None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:59:46.738517 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c00659-196b-459d-b0c3-75489c4d388f None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:59:46.738796 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c00659-196b-459d-b0c3-75489c4d388f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:59:46.739625 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c00659-196b-459d-b0c3-75489c4d388f None admin] RBAC: Adding query filter params (role.id=9db5671c65d3414ea2682afa2ccf1403, scope.project.id=f8652d1eff8d468eb8adeb5df6062ee5, user.id=0c62c5e7243b44488bb93d7b8dfef9b4) {{(pid=80540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 11:59:46.743404 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c00659-196b-459d-b0c3-75489c4d388f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:59:46.743404 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c00659-196b-459d-b0c3-75489c4d388f None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:59:46.747254 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 587/1161] 10.222.0.23 () {56 vars in 1374 bytes} [Thu May 19 11:59:46 2022] GET /identity/v3/role_assignments?user.id=0c62c5e7243b44488bb93d7b8dfef9b4&scope.project.id=f8652d1eff8d468eb8adeb5df6062ee5&role.id=9db5671c65d3414ea2682afa2ccf1403 => generated 608 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:59:48.983371 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39bca75-8946-4307-a876-c1ac57e3c42c None None] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:59:48.983856 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39bca75-8946-4307-a876-c1ac57e3c42c None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:59:48.984071 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39bca75-8946-4307-a876-c1ac57e3c42c None None] PATH_INFO: `/` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:59:48.985386 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 575/1162] 10.222.0.23 () {54 vars in 926 bytes} [Thu May 19 11:59:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 11:59:48.990715 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da012f5f-d6f1-475b-a47d-2c410c5200c3 None None] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:59:48.991132 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da012f5f-d6f1-475b-a47d-2c410c5200c3 None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:59:48.991275 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da012f5f-d6f1-475b-a47d-2c410c5200c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:59:49.008110 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.auth.core [None req-da012f5f-d6f1-475b-a47d-2c410c5200c3 None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 11:59:49.009304 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-da012f5f-d6f1-475b-a47d-2c410c5200c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 11:59:49.079809 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 588/1163] 10.222.0.23 () {58 vars in 1023 bytes} [Thu May 19 11:59:48 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 11:59:49.103748 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39bca75-8946-4307-a876-c1ac57e3c42c None None] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:59:49.116293 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:59:49.204662 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39bca75-8946-4307-a876-c1ac57e3c42c None None] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:59:49.238019 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270cb6cd-43a9-4872-a63e-cc43ad461517 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:59:49.238659 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270cb6cd-43a9-4872-a63e-cc43ad461517 None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:59:49.238949 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270cb6cd-43a9-4872-a63e-cc43ad461517 None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:59:49.239232 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270cb6cd-43a9-4872-a63e-cc43ad461517 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:59:49.304025 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270cb6cd-43a9-4872-a63e-cc43ad461517 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:59:49.305470 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270cb6cd-43a9-4872-a63e-cc43ad461517 None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:59:49.316631 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:59:49.402764 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 576/1164] 10.222.0.23 () {58 vars in 1282 bytes} [Thu May 19 11:59:49 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 11:59:49.409006 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da012f5f-d6f1-475b-a47d-2c410c5200c3 None None] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:59:49.506577 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da012f5f-d6f1-475b-a47d-2c410c5200c3 None None] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:59:49.517065 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:59:49.539579 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068141a5-6ef1-43d9-95c5-25cd3243ff8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:59:49.540405 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068141a5-6ef1-43d9-95c5-25cd3243ff8f None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:59:49.540693 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068141a5-6ef1-43d9-95c5-25cd3243ff8f None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:59:49.540969 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068141a5-6ef1-43d9-95c5-25cd3243ff8f None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:59:49.548255 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068141a5-6ef1-43d9-95c5-25cd3243ff8f None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:59:49.549844 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068141a5-6ef1-43d9-95c5-25cd3243ff8f None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:59:49.564435 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application [None req-068141a5-6ef1-43d9-95c5-25cd3243ff8f None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. May 19 11:59:49.564435 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 11:59:49.564435 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain May 19 11:59:49.564435 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) May 19 11:59:49.564435 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 11:59:49.564435 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 11:59:49.564435 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 11:59:49.564435 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 11:59:49.564435 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. May 19 11:59:49.564435 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application May 19 11:59:49.564435 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 11:59:49.564435 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application May 19 11:59:49.564435 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 11:59:49.564435 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 11:59:49.564435 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 11:59:49.564435 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 11:59:49.564435 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 11:59:49.565689 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 11:59:49.565689 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 11:59:49.565689 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 11:59:49.565689 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 11:59:49.565689 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 11:59:49.565689 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 11:59:49.565689 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 11:59:49.565689 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 11:59:49.565689 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 11:59:49.565689 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 11:59:49.565689 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 11:59:49.565689 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 11:59:49.565689 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 11:59:49.565689 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 11:59:49.565689 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 11:59:49.565689 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.get_or_create( May 19 11:59:49.566778 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 11:59:49.566778 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application with Lock( May 19 11:59:49.566778 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 11:59:49.566778 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._enter() May 19 11:59:49.566778 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 11:59:49.566778 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 11:59:49.566778 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 11:59:49.566778 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.creator() May 19 11:59:49.566778 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 11:59:49.566778 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application created_value = creator( May 19 11:59:49.566778 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain May 19 11:59:49.566778 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 11:59:49.566778 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. May 19 11:59:49.566778 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application May 19 11:59:49.567947 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 589/1165] 10.222.0.23 () {56 vars in 1117 bytes} [Thu May 19 11:59:49 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 11:59:49.571707 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270cb6cd-43a9-4872-a63e-cc43ad461517 None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:59:49.666711 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270cb6cd-43a9-4872-a63e-cc43ad461517 None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:59:49.677534 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:59:49.702337 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a8ef85-eab9-4449-a358-40364b44bb82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:59:49.702994 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a8ef85-eab9-4449-a358-40364b44bb82 None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:59:49.703280 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a8ef85-eab9-4449-a358-40364b44bb82 None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:59:49.703557 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a8ef85-eab9-4449-a358-40364b44bb82 None admin] PATH_INFO: `/v3/domains` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:59:49.704309 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a8ef85-eab9-4449-a358-40364b44bb82 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=80539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 11:59:49.704624 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a8ef85-eab9-4449-a358-40364b44bb82 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:59:49.706006 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a8ef85-eab9-4449-a358-40364b44bb82 None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:59:49.712935 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 577/1166] 10.222.0.23 () {56 vars in 1115 bytes} [Thu May 19 11:59:49 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:59:51.916881 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f0a807-38af-4c5d-82ca-b2930695169e None None] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:59:51.917443 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f0a807-38af-4c5d-82ca-b2930695169e None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:59:51.917443 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f0a807-38af-4c5d-82ca-b2930695169e None None] PATH_INFO: `/` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:59:51.918826 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 590/1167] 10.222.0.23 () {54 vars in 926 bytes} [Thu May 19 11:59:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 11:59:51.924823 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb58c394-a1c5-45f1-99bd-2cabf604f519 None None] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:59:51.925207 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb58c394-a1c5-45f1-99bd-2cabf604f519 None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:59:51.925343 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb58c394-a1c5-45f1-99bd-2cabf604f519 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:59:51.942886 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.auth.core [None req-cb58c394-a1c5-45f1-99bd-2cabf604f519 None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 11:59:51.944079 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-cb58c394-a1c5-45f1-99bd-2cabf604f519 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 11:59:52.009946 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 578/1168] 10.222.0.23 () {58 vars in 1023 bytes} [Thu May 19 11:59:51 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 11:59:52.033356 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f0a807-38af-4c5d-82ca-b2930695169e None None] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:59:52.140435 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f0a807-38af-4c5d-82ca-b2930695169e None None] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:59:52.174740 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caeb61c0-3d25-45a1-9212-45e5a15e89e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:59:52.175313 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caeb61c0-3d25-45a1-9212-45e5a15e89e5 None admin] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:59:52.175598 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caeb61c0-3d25-45a1-9212-45e5a15e89e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:59:52.175905 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caeb61c0-3d25-45a1-9212-45e5a15e89e5 None admin] PATH_INFO: `/v3/domains` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:59:52.176641 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caeb61c0-3d25-45a1-9212-45e5a15e89e5 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:59:52.178154 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caeb61c0-3d25-45a1-9212-45e5a15e89e5 None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:59:52.251119 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 591/1169] 10.222.0.23 () {60 vars in 1138 bytes} [Thu May 19 11:59:52 2022] POST /identity/v3/domains => generated 267 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 11:59:54.568222 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1976a09a-6557-4b08-aa07-208c8732f7e6 None None] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:59:54.568752 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1976a09a-6557-4b08-aa07-208c8732f7e6 None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:59:54.568752 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1976a09a-6557-4b08-aa07-208c8732f7e6 None None] PATH_INFO: `/` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:59:54.570128 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 579/1170] 10.222.0.23 () {54 vars in 926 bytes} [Thu May 19 11:59:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 11:59:54.575924 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c08d570-0bc1-4c3e-b433-12e7968c2acb None None] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:59:54.576386 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c08d570-0bc1-4c3e-b433-12e7968c2acb None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:59:54.576520 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c08d570-0bc1-4c3e-b433-12e7968c2acb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:59:54.595244 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.auth.core [None req-8c08d570-0bc1-4c3e-b433-12e7968c2acb None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 11:59:54.596479 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-8c08d570-0bc1-4c3e-b433-12e7968c2acb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 11:59:54.621489 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:59:54.668612 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 592/1171] 10.222.0.23 () {58 vars in 1023 bytes} [Thu May 19 11:59:54 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 11:59:54.692414 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1976a09a-6557-4b08-aa07-208c8732f7e6 None None] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:59:54.802584 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1976a09a-6557-4b08-aa07-208c8732f7e6 None None] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:59:54.806251 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:59:54.842131 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f314076-3921-4bec-8c3b-095e0b325a85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:59:54.842907 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f314076-3921-4bec-8c3b-095e0b325a85 None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:59:54.843184 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f314076-3921-4bec-8c3b-095e0b325a85 None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:59:54.843458 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f314076-3921-4bec-8c3b-095e0b325a85 None admin] PATH_INFO: `/v3/domains/62592962cb3a44dd9b7c8b9a5c30d11b` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:59:54.854392 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f314076-3921-4bec-8c3b-095e0b325a85 None admin] RBAC: Authorizing `identity:get_domain(domain_id=62592962cb3a44dd9b7c8b9a5c30d11b)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:59:54.855860 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f314076-3921-4bec-8c3b-095e0b325a85 None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:59:54.858228 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 580/1172] 10.222.0.23 () {56 vars in 1183 bytes} [Thu May 19 11:59:54 2022] GET /identity/v3/domains/62592962cb3a44dd9b7c8b9a5c30d11b => generated 267 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:59:54.864803 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c08d570-0bc1-4c3e-b433-12e7968c2acb None None] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:59:54.968006 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c08d570-0bc1-4c3e-b433-12e7968c2acb None None] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:59:55.002579 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02eed0e4-8d37-4449-8cc6-1b6c6727d082 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:59:55.003238 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02eed0e4-8d37-4449-8cc6-1b6c6727d082 None admin] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:59:55.003527 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02eed0e4-8d37-4449-8cc6-1b6c6727d082 None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:59:55.003893 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02eed0e4-8d37-4449-8cc6-1b6c6727d082 None admin] PATH_INFO: `/v3/projects` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:59:55.004556 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02eed0e4-8d37-4449-8cc6-1b6c6727d082 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:59:55.006004 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02eed0e4-8d37-4449-8cc6-1b6c6727d082 None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:59:55.086515 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 593/1173] 10.222.0.23 () {60 vars in 1141 bytes} [Thu May 19 11:59:54 2022] POST /identity/v3/projects => generated 361 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 11:59:57.379596 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721bef8a-198e-4be8-8fa5-80e99c4b0158 None None] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:59:57.380177 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721bef8a-198e-4be8-8fa5-80e99c4b0158 None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:59:57.380177 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721bef8a-198e-4be8-8fa5-80e99c4b0158 None None] PATH_INFO: `/` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:59:57.381543 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 581/1174] 10.222.0.23 () {54 vars in 926 bytes} [Thu May 19 11:59:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 11:59:57.386693 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58408dd-c404-448a-8104-7f13211100e4 None None] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:59:57.387111 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58408dd-c404-448a-8104-7f13211100e4 None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:59:57.387264 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58408dd-c404-448a-8104-7f13211100e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:59:57.404455 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.auth.core [None req-b58408dd-c404-448a-8104-7f13211100e4 None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 11:59:57.405643 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-b58408dd-c404-448a-8104-7f13211100e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 11:59:57.413870 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:59:57.474095 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 594/1175] 10.222.0.23 () {58 vars in 1023 bytes} [Thu May 19 11:59:57 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 11:59:57.498709 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721bef8a-198e-4be8-8fa5-80e99c4b0158 None None] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:59:57.598094 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721bef8a-198e-4be8-8fa5-80e99c4b0158 None None] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:59:57.631274 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c367e2-1677-43ee-81b3-41f37262f3c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:59:57.632073 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c367e2-1677-43ee-81b3-41f37262f3c8 None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:59:57.632357 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c367e2-1677-43ee-81b3-41f37262f3c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:59:57.632631 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c367e2-1677-43ee-81b3-41f37262f3c8 None admin] PATH_INFO: `/v3/domains/62592962cb3a44dd9b7c8b9a5c30d11b` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:59:57.633799 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c367e2-1677-43ee-81b3-41f37262f3c8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=62592962cb3a44dd9b7c8b9a5c30d11b)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:59:57.635254 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c367e2-1677-43ee-81b3-41f37262f3c8 None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:59:57.636884 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 582/1176] 10.222.0.23 () {56 vars in 1183 bytes} [Thu May 19 11:59:57 2022] GET /identity/v3/domains/62592962cb3a44dd9b7c8b9a5c30d11b => generated 267 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 11:59:57.643109 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58408dd-c404-448a-8104-7f13211100e4 None None] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 11:59:57.741830 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58408dd-c404-448a-8104-7f13211100e4 None None] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 11:59:57.775892 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f78fe7c-fd10-409c-9189-cec7dbd9ce6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 11:59:57.776564 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f78fe7c-fd10-409c-9189-cec7dbd9ce6a None admin] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 11:59:57.776879 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f78fe7c-fd10-409c-9189-cec7dbd9ce6a None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 11:59:57.777166 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f78fe7c-fd10-409c-9189-cec7dbd9ce6a None admin] PATH_INFO: `/v3/users` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 11:59:57.777873 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f78fe7c-fd10-409c-9189-cec7dbd9ce6a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 11:59:57.780194 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f78fe7c-fd10-409c-9189-cec7dbd9ce6a None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 11:59:57.874689 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 11:59:57.880759 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 595/1177] 10.222.0.23 () {60 vars in 1132 bytes} [Thu May 19 11:59:57 2022] POST /identity/v3/users => generated 311 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 12:00:00.149742 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524f16f8-3343-42a8-86d4-c8a671d0e80c None None] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:00:00.150293 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524f16f8-3343-42a8-86d4-c8a671d0e80c None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:00:00.150293 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524f16f8-3343-42a8-86d4-c8a671d0e80c None None] PATH_INFO: `/` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:00:00.151653 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 583/1178] 10.222.0.23 () {54 vars in 926 bytes} [Thu May 19 12:00:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 12:00:00.157242 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee99252e-5821-4b52-8580-07b8142592fb None None] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:00:00.157678 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee99252e-5821-4b52-8580-07b8142592fb None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:00:00.157816 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee99252e-5821-4b52-8580-07b8142592fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:00:00.174847 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.auth.core [None req-ee99252e-5821-4b52-8580-07b8142592fb None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:00:00.176083 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-ee99252e-5821-4b52-8580-07b8142592fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:00:00.246141 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 596/1179] 10.222.0.23 () {58 vars in 1023 bytes} [Thu May 19 12:00:00 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 12:00:00.269433 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524f16f8-3343-42a8-86d4-c8a671d0e80c None None] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:00:00.301867 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:00:00.372002 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524f16f8-3343-42a8-86d4-c8a671d0e80c None None] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:00:00.404452 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6a7154-294a-4b07-86f1-caf6232a53e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:00:00.405222 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6a7154-294a-4b07-86f1-caf6232a53e6 None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:00:00.405505 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6a7154-294a-4b07-86f1-caf6232a53e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:00:00.405781 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6a7154-294a-4b07-86f1-caf6232a53e6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:00:00.416071 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6a7154-294a-4b07-86f1-caf6232a53e6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 12:00:00.424188 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application [None req-ab6a7154-294a-4b07-86f1-caf6232a53e6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 12:00:00.424188 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:00:00.424188 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 12:00:00.424188 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 12:00:00.424188 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 12:00:00.424188 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 12:00:00.424188 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 12:00:00.424188 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 12:00:00.424188 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 12:00:00.424188 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 12:00:00.424188 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 12:00:00.424188 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 12:00:00.424188 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 12:00:00.424188 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 12:00:00.424188 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 12:00:00.424188 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 12:00:00.425281 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 12:00:00.425281 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 12:00:00.425281 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 12:00:00.425281 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 12:00:00.425281 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 12:00:00.425281 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 12:00:00.425281 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 12:00:00.425281 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 12:00:00.425281 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 12:00:00.425281 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.get_or_create( May 19 12:00:00.425281 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 12:00:00.425281 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application with Lock( May 19 12:00:00.425281 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 12:00:00.425281 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._enter() May 19 12:00:00.425281 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 12:00:00.425281 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 12:00:00.426120 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 12:00:00.426120 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.creator() May 19 12:00:00.426120 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 12:00:00.426120 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application created_value = creator( May 19 12:00:00.426120 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 12:00:00.426120 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 12:00:00.426120 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 12:00:00.426120 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 12:00:00.426120 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 12:00:00.426120 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 12:00:00.426120 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 12:00:00.426120 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application May 19 12:00:00.426120 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 584/1180] 10.222.0.23 () {56 vars in 1096 bytes} [Thu May 19 12:00:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 12:00:00.431372 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee99252e-5821-4b52-8580-07b8142592fb None None] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:00:00.535993 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee99252e-5821-4b52-8580-07b8142592fb None None] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:00:00.561877 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:00:00.569497 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3350088-dd55-449e-b93f-d3d121220851 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:00:00.571232 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3350088-dd55-449e-b93f-d3d121220851 None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:00:00.571232 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3350088-dd55-449e-b93f-d3d121220851 None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:00:00.571232 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3350088-dd55-449e-b93f-d3d121220851 None admin] PATH_INFO: `/v3/roles` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:00:00.571511 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3350088-dd55-449e-b93f-d3d121220851 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:00:00.571847 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3350088-dd55-449e-b93f-d3d121220851 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:00:00.573502 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3350088-dd55-449e-b93f-d3d121220851 None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:00:00.581556 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 597/1181] 10.222.0.23 () {56 vars in 1099 bytes} [Thu May 19 12:00:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:00:00.587361 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6a7154-294a-4b07-86f1-caf6232a53e6 None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:00:00.676460 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:00:00.684764 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6a7154-294a-4b07-86f1-caf6232a53e6 None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:00:00.719176 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340e31a6-507e-4033-b4ff-4d6b42aed22f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:00:00.724374 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340e31a6-507e-4033-b4ff-4d6b42aed22f None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:00:00.724374 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340e31a6-507e-4033-b4ff-4d6b42aed22f None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:00:00.724374 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340e31a6-507e-4033-b4ff-4d6b42aed22f None admin] PATH_INFO: `/v3/users/d5bd08b7fad5410790992abec684e137` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:00:00.734409 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-340e31a6-507e-4033-b4ff-4d6b42aed22f None admin] RBAC: Authorizing `identity:get_user(user_id=d5bd08b7fad5410790992abec684e137)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:00:00.735921 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-340e31a6-507e-4033-b4ff-4d6b42aed22f None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:00:00.737672 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 585/1182] 10.222.0.23 () {56 vars in 1177 bytes} [Thu May 19 12:00:00 2022] GET /identity/v3/users/d5bd08b7fad5410790992abec684e137 => generated 311 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:00:00.743442 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3350088-dd55-449e-b93f-d3d121220851 None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:00:00.843604 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3350088-dd55-449e-b93f-d3d121220851 None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:00:00.877592 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85481fe9-b128-45c3-8faa-bc3d5d027d36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:00:00.878233 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85481fe9-b128-45c3-8faa-bc3d5d027d36 None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:00:00.878522 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85481fe9-b128-45c3-8faa-bc3d5d027d36 None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:00:00.878803 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85481fe9-b128-45c3-8faa-bc3d5d027d36 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:00:00.934230 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:00:00.944988 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85481fe9-b128-45c3-8faa-bc3d5d027d36 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:00:00.946473 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85481fe9-b128-45c3-8faa-bc3d5d027d36 None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:00:01.044724 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 598/1183] 10.222.0.23 () {58 vars in 1282 bytes} [Thu May 19 12:00:00 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 12:00:01.050647 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340e31a6-507e-4033-b4ff-4d6b42aed22f None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:00:01.148413 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340e31a6-507e-4033-b4ff-4d6b42aed22f None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:00:01.180177 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e785b228-8aa4-4f44-8a92-32fce64d35b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:00:01.181030 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e785b228-8aa4-4f44-8a92-32fce64d35b0 None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:00:01.181313 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e785b228-8aa4-4f44-8a92-32fce64d35b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:00:01.181589 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e785b228-8aa4-4f44-8a92-32fce64d35b0 None admin] PATH_INFO: `/v3/projects/25cc60ca0e074d5e86975872d5d35872` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:00:01.182753 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e785b228-8aa4-4f44-8a92-32fce64d35b0 None admin] RBAC: Authorizing `identity:get_project(project_id=25cc60ca0e074d5e86975872d5d35872)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:00:01.184221 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e785b228-8aa4-4f44-8a92-32fce64d35b0 None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:00:01.186415 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 586/1184] 10.222.0.23 () {56 vars in 1186 bytes} [Thu May 19 12:00:01 2022] GET /identity/v3/projects/25cc60ca0e074d5e86975872d5d35872 => generated 361 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:00:01.192441 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85481fe9-b128-45c3-8faa-bc3d5d027d36 None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:00:01.279247 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:00:01.288222 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85481fe9-b128-45c3-8faa-bc3d5d027d36 None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:00:01.320935 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc03b2f-ea38-4f1f-bf72-ef816020c1ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:00:01.321592 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc03b2f-ea38-4f1f-bf72-ef816020c1ed None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:00:01.322079 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc03b2f-ea38-4f1f-bf72-ef816020c1ed None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:00:01.322584 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc03b2f-ea38-4f1f-bf72-ef816020c1ed None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:00:01.324074 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc03b2f-ea38-4f1f-bf72-ef816020c1ed None admin] RBAC: Adding query filter params (role.id=9db5671c65d3414ea2682afa2ccf1403, scope.project.id=25cc60ca0e074d5e86975872d5d35872, user.id=d5bd08b7fad5410790992abec684e137) {{(pid=80540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:00:01.324401 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc03b2f-ea38-4f1f-bf72-ef816020c1ed None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:00:01.326322 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc03b2f-ea38-4f1f-bf72-ef816020c1ed None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:00:01.331999 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 599/1185] 10.222.0.23 () {56 vars in 1374 bytes} [Thu May 19 12:00:01 2022] GET /identity/v3/role_assignments?user.id=d5bd08b7fad5410790992abec684e137&scope.project.id=25cc60ca0e074d5e86975872d5d35872&role.id=9db5671c65d3414ea2682afa2ccf1403 => generated 259 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:00:03.507804 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d65dd90-053a-4456-828e-6c8ab37ac49a None None] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:00:03.508360 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d65dd90-053a-4456-828e-6c8ab37ac49a None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:00:03.508504 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d65dd90-053a-4456-828e-6c8ab37ac49a None None] PATH_INFO: `/` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:00:03.509880 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 587/1186] 10.222.0.23 () {54 vars in 926 bytes} [Thu May 19 12:00:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 12:00:03.515936 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea53f786-276f-4fc7-b6a3-18e152e55c65 None None] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:00:03.516357 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea53f786-276f-4fc7-b6a3-18e152e55c65 None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:00:03.516496 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea53f786-276f-4fc7-b6a3-18e152e55c65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:00:03.533439 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.auth.core [None req-ea53f786-276f-4fc7-b6a3-18e152e55c65 None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:00:03.534625 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-ea53f786-276f-4fc7-b6a3-18e152e55c65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:00:03.600586 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 600/1187] 10.222.0.23 () {58 vars in 1023 bytes} [Thu May 19 12:00:03 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 12:00:03.624018 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d65dd90-053a-4456-828e-6c8ab37ac49a None None] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:00:03.643583 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:00:03.723943 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d65dd90-053a-4456-828e-6c8ab37ac49a None None] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:00:03.755939 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553dde96-d15e-495a-963b-2c6e0c8b3530 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:00:03.757364 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553dde96-d15e-495a-963b-2c6e0c8b3530 None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:00:03.757364 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553dde96-d15e-495a-963b-2c6e0c8b3530 None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:00:03.757364 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553dde96-d15e-495a-963b-2c6e0c8b3530 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:00:03.769883 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-553dde96-d15e-495a-963b-2c6e0c8b3530 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 12:00:03.777361 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application [None req-553dde96-d15e-495a-963b-2c6e0c8b3530 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 12:00:03.777361 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:00:03.777361 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 12:00:03.777361 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 12:00:03.777361 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 12:00:03.777361 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 12:00:03.777361 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 12:00:03.777361 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 12:00:03.777361 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 12:00:03.777361 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 12:00:03.777361 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 12:00:03.777361 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 12:00:03.777361 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 12:00:03.777361 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 12:00:03.777361 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 12:00:03.777361 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 12:00:03.779253 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 12:00:03.779253 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 12:00:03.779253 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 12:00:03.779253 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 12:00:03.779253 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 12:00:03.779253 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 12:00:03.779253 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 12:00:03.779253 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 12:00:03.779253 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 12:00:03.779253 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.get_or_create( May 19 12:00:03.779253 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 12:00:03.779253 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application with Lock( May 19 12:00:03.779253 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 12:00:03.779253 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._enter() May 19 12:00:03.779253 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 12:00:03.779253 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 12:00:03.780157 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 12:00:03.780157 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.creator() May 19 12:00:03.780157 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 12:00:03.780157 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application created_value = creator( May 19 12:00:03.780157 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 12:00:03.780157 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 12:00:03.780157 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 12:00:03.780157 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 12:00:03.780157 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 12:00:03.780157 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 12:00:03.780157 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 12:00:03.780157 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application May 19 12:00:03.780157 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 588/1188] 10.222.0.23 () {56 vars in 1096 bytes} [Thu May 19 12:00:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 12:00:03.785340 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea53f786-276f-4fc7-b6a3-18e152e55c65 None None] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:00:03.871180 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:00:03.886464 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea53f786-276f-4fc7-b6a3-18e152e55c65 None None] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:00:03.920129 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dec61e6-25d7-4c60-b748-edbfb2a13dbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:00:03.920780 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dec61e6-25d7-4c60-b748-edbfb2a13dbb None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:00:03.921066 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dec61e6-25d7-4c60-b748-edbfb2a13dbb None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:00:03.921343 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dec61e6-25d7-4c60-b748-edbfb2a13dbb None admin] PATH_INFO: `/v3/roles` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:00:03.922081 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dec61e6-25d7-4c60-b748-edbfb2a13dbb None admin] RBAC: Adding query filter params (name=admin) {{(pid=80540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:00:03.922388 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dec61e6-25d7-4c60-b748-edbfb2a13dbb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:00:03.923780 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dec61e6-25d7-4c60-b748-edbfb2a13dbb None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:00:03.931964 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 601/1189] 10.222.0.23 () {56 vars in 1099 bytes} [Thu May 19 12:00:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:00:03.937813 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553dde96-d15e-495a-963b-2c6e0c8b3530 None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:00:04.014900 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:00:04.035245 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553dde96-d15e-495a-963b-2c6e0c8b3530 None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:00:04.068922 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c73e279-2ca2-4707-9390-ef1d8be7a77b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:00:04.069781 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c73e279-2ca2-4707-9390-ef1d8be7a77b None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:00:04.070065 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c73e279-2ca2-4707-9390-ef1d8be7a77b None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:00:04.070339 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c73e279-2ca2-4707-9390-ef1d8be7a77b None admin] PATH_INFO: `/v3/users/d5bd08b7fad5410790992abec684e137` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:00:04.071506 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c73e279-2ca2-4707-9390-ef1d8be7a77b None admin] RBAC: Authorizing `identity:get_user(user_id=d5bd08b7fad5410790992abec684e137)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:00:04.073212 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c73e279-2ca2-4707-9390-ef1d8be7a77b None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:00:04.074830 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 589/1190] 10.222.0.23 () {56 vars in 1177 bytes} [Thu May 19 12:00:03 2022] GET /identity/v3/users/d5bd08b7fad5410790992abec684e137 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:00:04.081148 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dec61e6-25d7-4c60-b748-edbfb2a13dbb None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:00:04.177879 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dec61e6-25d7-4c60-b748-edbfb2a13dbb None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:00:04.211509 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35e6286-3600-499c-8567-bcb14bf24466 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:00:04.212455 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35e6286-3600-499c-8567-bcb14bf24466 None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:00:04.212740 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35e6286-3600-499c-8567-bcb14bf24466 None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:00:04.213017 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35e6286-3600-499c-8567-bcb14bf24466 None admin] PATH_INFO: `/v3/projects/25cc60ca0e074d5e86975872d5d35872` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:00:04.214176 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35e6286-3600-499c-8567-bcb14bf24466 None admin] RBAC: Authorizing `identity:get_project(project_id=25cc60ca0e074d5e86975872d5d35872)` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:00:04.215618 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35e6286-3600-499c-8567-bcb14bf24466 None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:00:04.217922 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 602/1191] 10.222.0.23 () {56 vars in 1186 bytes} [Thu May 19 12:00:04 2022] GET /identity/v3/projects/25cc60ca0e074d5e86975872d5d35872 => generated 361 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:00:04.223986 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c73e279-2ca2-4707-9390-ef1d8be7a77b None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:00:04.322382 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c73e279-2ca2-4707-9390-ef1d8be7a77b None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:00:04.355397 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0e7f3e-6d9e-4caa-97d8-5fe811c17dcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:00:04.356366 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0e7f3e-6d9e-4caa-97d8-5fe811c17dcd None admin] REQUEST_METHOD: `PUT` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:00:04.357161 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0e7f3e-6d9e-4caa-97d8-5fe811c17dcd None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:00:04.357434 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0e7f3e-6d9e-4caa-97d8-5fe811c17dcd None admin] PATH_INFO: `/v3/projects/25cc60ca0e074d5e86975872d5d35872/users/d5bd08b7fad5410790992abec684e137/roles/9db5671c65d3414ea2682afa2ccf1403` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:00:04.359367 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0e7f3e-6d9e-4caa-97d8-5fe811c17dcd None admin] RBAC: Authorizing `identity:create_grant(project_id=25cc60ca0e074d5e86975872d5d35872, user_id=d5bd08b7fad5410790992abec684e137, role_id=9db5671c65d3414ea2682afa2ccf1403)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:00:04.360839 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0e7f3e-6d9e-4caa-97d8-5fe811c17dcd None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:00:04.445145 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 590/1192] 10.222.0.23 () {58 vars in 1439 bytes} [Thu May 19 12:00:04 2022] PUT /identity/v3/projects/25cc60ca0e074d5e86975872d5d35872/users/d5bd08b7fad5410790992abec684e137/roles/9db5671c65d3414ea2682afa2ccf1403 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:00:06.635733 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5036cd6-13b4-4e64-8ef3-3634e8c5c445 None None] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:00:06.636376 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5036cd6-13b4-4e64-8ef3-3634e8c5c445 None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:00:06.636376 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5036cd6-13b4-4e64-8ef3-3634e8c5c445 None None] PATH_INFO: `/` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:00:06.638299 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 603/1193] 10.222.0.23 () {54 vars in 926 bytes} [Thu May 19 12:00:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 12:00:06.643567 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91a3164-fdb5-42eb-91d6-cb481eb9206d None None] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:00:06.644045 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91a3164-fdb5-42eb-91d6-cb481eb9206d None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:00:06.644156 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91a3164-fdb5-42eb-91d6-cb481eb9206d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:00:06.655604 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:00:06.661401 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.auth.core [None req-c91a3164-fdb5-42eb-91d6-cb481eb9206d None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:00:06.662593 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-c91a3164-fdb5-42eb-91d6-cb481eb9206d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:00:06.728400 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 591/1194] 10.222.0.23 () {58 vars in 1023 bytes} [Thu May 19 12:00:06 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 12:00:06.751350 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5036cd6-13b4-4e64-8ef3-3634e8c5c445 None None] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:00:06.768229 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:00:06.852939 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5036cd6-13b4-4e64-8ef3-3634e8c5c445 None None] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:00:06.886935 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b3436c-8af6-4825-a165-57fdcc003d8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:00:06.887648 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b3436c-8af6-4825-a165-57fdcc003d8a None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:00:06.888477 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b3436c-8af6-4825-a165-57fdcc003d8a None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:00:06.888756 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b3436c-8af6-4825-a165-57fdcc003d8a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:00:06.898680 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b3436c-8af6-4825-a165-57fdcc003d8a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 12:00:06.906816 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application [None req-73b3436c-8af6-4825-a165-57fdcc003d8a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 12:00:06.906816 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:00:06.906816 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 12:00:06.906816 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 12:00:06.906816 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 12:00:06.906816 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 12:00:06.906816 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 12:00:06.906816 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 12:00:06.906816 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 12:00:06.906816 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 12:00:06.906816 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 12:00:06.906816 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 12:00:06.906816 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 12:00:06.906816 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 12:00:06.906816 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 12:00:06.906816 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 12:00:06.908049 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 12:00:06.908049 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 12:00:06.908049 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 12:00:06.908049 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 12:00:06.908049 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 12:00:06.908049 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 12:00:06.908049 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 12:00:06.908049 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 12:00:06.908049 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 12:00:06.908049 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.get_or_create( May 19 12:00:06.908049 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 12:00:06.908049 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application with Lock( May 19 12:00:06.908049 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 12:00:06.908049 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._enter() May 19 12:00:06.908049 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 12:00:06.908049 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 12:00:06.909149 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 12:00:06.909149 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.creator() May 19 12:00:06.909149 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 12:00:06.909149 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application created_value = creator( May 19 12:00:06.909149 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 12:00:06.909149 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 12:00:06.909149 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 12:00:06.909149 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 12:00:06.909149 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 12:00:06.909149 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 12:00:06.909149 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 12:00:06.909149 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application May 19 12:00:06.909149 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 604/1195] 10.222.0.23 () {56 vars in 1096 bytes} [Thu May 19 12:00:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 12:00:06.914068 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91a3164-fdb5-42eb-91d6-cb481eb9206d None None] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:00:07.011906 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91a3164-fdb5-42eb-91d6-cb481eb9206d None None] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:00:07.046114 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65513fea-79a2-42c4-a42c-eebe68853d52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:00:07.046767 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65513fea-79a2-42c4-a42c-eebe68853d52 None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:00:07.047054 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65513fea-79a2-42c4-a42c-eebe68853d52 None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:00:07.047329 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65513fea-79a2-42c4-a42c-eebe68853d52 None admin] PATH_INFO: `/v3/roles` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:00:07.048101 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65513fea-79a2-42c4-a42c-eebe68853d52 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:00:07.048406 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65513fea-79a2-42c4-a42c-eebe68853d52 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:00:07.049775 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65513fea-79a2-42c4-a42c-eebe68853d52 None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:00:07.051932 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:00:07.058168 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 592/1196] 10.222.0.23 () {56 vars in 1099 bytes} [Thu May 19 12:00:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:00:07.064116 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b3436c-8af6-4825-a165-57fdcc003d8a None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:00:07.139304 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:00:07.163751 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b3436c-8af6-4825-a165-57fdcc003d8a None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:00:07.197301 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244cabe7-9790-4a84-b933-0ef85c21aef4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:00:07.198420 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244cabe7-9790-4a84-b933-0ef85c21aef4 None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:00:07.198707 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244cabe7-9790-4a84-b933-0ef85c21aef4 None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:00:07.198987 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244cabe7-9790-4a84-b933-0ef85c21aef4 None admin] PATH_INFO: `/v3/users/d5bd08b7fad5410790992abec684e137` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:00:07.200190 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244cabe7-9790-4a84-b933-0ef85c21aef4 None admin] RBAC: Authorizing `identity:get_user(user_id=d5bd08b7fad5410790992abec684e137)` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:00:07.201673 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244cabe7-9790-4a84-b933-0ef85c21aef4 None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:00:07.203297 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 605/1197] 10.222.0.23 () {56 vars in 1177 bytes} [Thu May 19 12:00:07 2022] GET /identity/v3/users/d5bd08b7fad5410790992abec684e137 => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:00:07.209003 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65513fea-79a2-42c4-a42c-eebe68853d52 None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:00:07.306998 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65513fea-79a2-42c4-a42c-eebe68853d52 None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:00:07.339009 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8112c678-3269-4274-bdbd-de5e3b87611f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:00:07.339646 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8112c678-3269-4274-bdbd-de5e3b87611f None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:00:07.339982 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8112c678-3269-4274-bdbd-de5e3b87611f None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:00:07.340258 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8112c678-3269-4274-bdbd-de5e3b87611f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:00:07.402123 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:00:07.406021 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8112c678-3269-4274-bdbd-de5e3b87611f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:00:07.407459 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8112c678-3269-4274-bdbd-de5e3b87611f None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:00:07.501648 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 593/1198] 10.222.0.23 () {58 vars in 1282 bytes} [Thu May 19 12:00:07 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 12:00:07.507496 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244cabe7-9790-4a84-b933-0ef85c21aef4 None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:00:07.606306 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244cabe7-9790-4a84-b933-0ef85c21aef4 None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:00:07.638925 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9271f6-5882-4600-886d-e0791ccf3410 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:00:07.639775 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9271f6-5882-4600-886d-e0791ccf3410 None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:00:07.640093 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9271f6-5882-4600-886d-e0791ccf3410 None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:00:07.640372 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9271f6-5882-4600-886d-e0791ccf3410 None admin] PATH_INFO: `/v3/projects/25cc60ca0e074d5e86975872d5d35872` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:00:07.641555 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9271f6-5882-4600-886d-e0791ccf3410 None admin] RBAC: Authorizing `identity:get_project(project_id=25cc60ca0e074d5e86975872d5d35872)` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:00:07.643013 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9271f6-5882-4600-886d-e0791ccf3410 None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:00:07.644757 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 606/1199] 10.222.0.23 () {56 vars in 1186 bytes} [Thu May 19 12:00:07 2022] GET /identity/v3/projects/25cc60ca0e074d5e86975872d5d35872 => generated 361 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:00:07.650775 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8112c678-3269-4274-bdbd-de5e3b87611f None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:00:07.743870 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:00:07.748059 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8112c678-3269-4274-bdbd-de5e3b87611f None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:00:07.781756 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6368082-9992-4ac0-b0c7-6242d15215fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:00:07.782407 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6368082-9992-4ac0-b0c7-6242d15215fe None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:00:07.782692 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6368082-9992-4ac0-b0c7-6242d15215fe None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:00:07.782968 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6368082-9992-4ac0-b0c7-6242d15215fe None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:00:07.783817 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6368082-9992-4ac0-b0c7-6242d15215fe None admin] RBAC: Adding query filter params (role.id=9db5671c65d3414ea2682afa2ccf1403, scope.project.id=25cc60ca0e074d5e86975872d5d35872, user.id=d5bd08b7fad5410790992abec684e137) {{(pid=80539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:00:07.784138 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6368082-9992-4ac0-b0c7-6242d15215fe None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:00:07.785565 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6368082-9992-4ac0-b0c7-6242d15215fe None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:00:07.791268 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 594/1200] 10.222.0.23 () {56 vars in 1374 bytes} [Thu May 19 12:00:07 2022] GET /identity/v3/role_assignments?user.id=d5bd08b7fad5410790992abec684e137&scope.project.id=25cc60ca0e074d5e86975872d5d35872&role.id=9db5671c65d3414ea2682afa2ccf1403 => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:00:17.634221 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=95 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:00:17.788658 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=19 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:01:26.066137 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb48e4b6-4d8c-4792-9654-317df3e2f54e None None] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:26.066715 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb48e4b6-4d8c-4792-9654-317df3e2f54e None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:26.066715 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb48e4b6-4d8c-4792-9654-317df3e2f54e None None] PATH_INFO: `/` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:26.068163 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 607/1201] 10.222.0.23 () {54 vars in 926 bytes} [Thu May 19 12:01:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 12:01:26.073673 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f219f48-e34f-4666-a109-2b034f69a91a None None] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:26.074096 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f219f48-e34f-4666-a109-2b034f69a91a None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:26.074234 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f219f48-e34f-4666-a109-2b034f69a91a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:26.091243 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.auth.core [None req-1f219f48-e34f-4666-a109-2b034f69a91a None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:01:26.093268 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-1f219f48-e34f-4666-a109-2b034f69a91a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:01:26.164788 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 595/1202] 10.222.0.23 () {58 vars in 1023 bytes} [Thu May 19 12:01:26 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 12:01:26.191040 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb48e4b6-4d8c-4792-9654-317df3e2f54e None None] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:01:26.296577 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb48e4b6-4d8c-4792-9654-317df3e2f54e None None] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:01:26.331376 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4e9217-7bdd-4676-a830-e7749c366541 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:01:26.332342 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4e9217-7bdd-4676-a830-e7749c366541 None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:26.332626 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4e9217-7bdd-4676-a830-e7749c366541 None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:26.332901 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4e9217-7bdd-4676-a830-e7749c366541 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:26.343759 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4e9217-7bdd-4676-a830-e7749c366541 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:01:26.345250 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4e9217-7bdd-4676-a830-e7749c366541 None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:01:26.362803 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application [None req-6b4e9217-7bdd-4676-a830-e7749c366541 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 12:01:26.362803 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:01:26.362803 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 12:01:26.362803 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 12:01:26.362803 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 12:01:26.362803 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 12:01:26.362803 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 12:01:26.362803 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 12:01:26.362803 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 12:01:26.362803 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 12:01:26.362803 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 12:01:26.362803 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 12:01:26.362803 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 12:01:26.362803 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 12:01:26.362803 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 12:01:26.364015 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 12:01:26.364015 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 12:01:26.364015 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 12:01:26.364015 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 12:01:26.364015 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 12:01:26.364015 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 12:01:26.364015 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.get_or_create( May 19 12:01:26.364015 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 12:01:26.364015 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application with Lock( May 19 12:01:26.364015 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 12:01:26.364015 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._enter() May 19 12:01:26.364015 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 12:01:26.364015 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 12:01:26.364015 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 12:01:26.364015 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.creator() May 19 12:01:26.364015 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 12:01:26.364015 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application created_value = creator( May 19 12:01:26.365080 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 12:01:26.365080 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 12:01:26.365080 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 12:01:26.365080 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application May 19 12:01:26.365080 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 608/1203] 10.222.0.23 () {56 vars in 1108 bytes} [Thu May 19 12:01:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 12:01:26.370049 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f219f48-e34f-4666-a109-2b034f69a91a None None] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:01:26.474453 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f219f48-e34f-4666-a109-2b034f69a91a None None] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:01:26.505067 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:01:26.510820 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1784663-fb15-4ee2-93a1-aeaed9430da0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:01:26.511500 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1784663-fb15-4ee2-93a1-aeaed9430da0 None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:26.511779 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1784663-fb15-4ee2-93a1-aeaed9430da0 None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:26.512080 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1784663-fb15-4ee2-93a1-aeaed9430da0 None admin] PATH_INFO: `/v3/domains` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:26.512804 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1784663-fb15-4ee2-93a1-aeaed9430da0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:01:26.513831 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1784663-fb15-4ee2-93a1-aeaed9430da0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:01:26.515208 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1784663-fb15-4ee2-93a1-aeaed9430da0 None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:01:26.526873 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 596/1204] 10.222.0.23 () {56 vars in 1109 bytes} [Thu May 19 12:01:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:01:26.532693 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4e9217-7bdd-4676-a830-e7749c366541 None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:01:26.584811 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:01:26.630387 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4e9217-7bdd-4676-a830-e7749c366541 None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:01:26.665883 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850eb76b-f2c6-47a9-8fc2-7012eabe0ce4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:01:26.666604 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850eb76b-f2c6-47a9-8fc2-7012eabe0ce4 None admin] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:26.667350 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850eb76b-f2c6-47a9-8fc2-7012eabe0ce4 None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:26.667662 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850eb76b-f2c6-47a9-8fc2-7012eabe0ce4 None admin] PATH_INFO: `/v3/users` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:26.668445 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850eb76b-f2c6-47a9-8fc2-7012eabe0ce4 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:01:26.670528 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850eb76b-f2c6-47a9-8fc2-7012eabe0ce4 None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:01:26.740070 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 609/1205] 10.222.0.23 () {60 vars in 1132 bytes} [Thu May 19 12:01:26 2022] POST /identity/v3/users => generated 251 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 12:01:29.041170 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bfde7b-bf45-46a5-a5d6-f14a20746bda None None] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:29.041716 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bfde7b-bf45-46a5-a5d6-f14a20746bda None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:29.041716 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bfde7b-bf45-46a5-a5d6-f14a20746bda None None] PATH_INFO: `/` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:29.043071 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 597/1206] 10.222.0.23 () {54 vars in 926 bytes} [Thu May 19 12:01:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 12:01:29.048938 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8e40ca-8a7f-4497-80fa-77b04e9ad913 None None] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:29.049370 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8e40ca-8a7f-4497-80fa-77b04e9ad913 None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:29.049501 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8e40ca-8a7f-4497-80fa-77b04e9ad913 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:29.067063 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.auth.core [None req-ce8e40ca-8a7f-4497-80fa-77b04e9ad913 None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:01:29.068754 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-ce8e40ca-8a7f-4497-80fa-77b04e9ad913 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:01:29.136741 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 610/1207] 10.222.0.23 () {58 vars in 1023 bytes} [Thu May 19 12:01:29 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 12:01:29.160949 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98bfde7b-bf45-46a5-a5d6-f14a20746bda None None] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:01:29.260457 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98bfde7b-bf45-46a5-a5d6-f14a20746bda None None] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:01:29.292967 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf90af4a-9878-4a34-a703-e3cbb6d5999c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:01:29.293746 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf90af4a-9878-4a34-a703-e3cbb6d5999c None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:29.294027 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf90af4a-9878-4a34-a703-e3cbb6d5999c None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:29.294828 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf90af4a-9878-4a34-a703-e3cbb6d5999c None admin] PATH_INFO: `/v3/roles/service` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:29.304807 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf90af4a-9878-4a34-a703-e3cbb6d5999c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 12:01:29.312582 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application [None req-bf90af4a-9878-4a34-a703-e3cbb6d5999c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 12:01:29.312582 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:01:29.312582 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 12:01:29.312582 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 12:01:29.312582 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 12:01:29.312582 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 12:01:29.312582 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 12:01:29.312582 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 12:01:29.312582 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 12:01:29.312582 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 12:01:29.312582 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 12:01:29.312582 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 12:01:29.312582 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 12:01:29.312582 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 12:01:29.312582 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 12:01:29.312582 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 12:01:29.313727 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 12:01:29.313727 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 12:01:29.313727 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 12:01:29.313727 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 12:01:29.313727 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 12:01:29.313727 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 12:01:29.313727 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 12:01:29.313727 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 12:01:29.313727 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 12:01:29.313727 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.get_or_create( May 19 12:01:29.313727 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 12:01:29.313727 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application with Lock( May 19 12:01:29.313727 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 12:01:29.313727 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._enter() May 19 12:01:29.313727 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 12:01:29.313727 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 12:01:29.314965 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 12:01:29.314965 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.creator() May 19 12:01:29.314965 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 12:01:29.314965 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application created_value = creator( May 19 12:01:29.314965 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 12:01:29.314965 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 12:01:29.314965 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 12:01:29.314965 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 12:01:29.314965 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 12:01:29.314965 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 12:01:29.314965 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 12:01:29.314965 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application May 19 12:01:29.314965 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 598/1208] 10.222.0.23 () {56 vars in 1102 bytes} [Thu May 19 12:01:29 2022] GET /identity/v3/roles/service => generated 85 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 12:01:29.319749 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8e40ca-8a7f-4497-80fa-77b04e9ad913 None None] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:01:29.358993 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:01:29.415195 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8e40ca-8a7f-4497-80fa-77b04e9ad913 None None] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:01:29.449052 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3908b570-fcb6-46cb-b28e-0f9a996b0904 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:01:29.449715 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3908b570-fcb6-46cb-b28e-0f9a996b0904 None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:29.450002 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3908b570-fcb6-46cb-b28e-0f9a996b0904 None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:29.450288 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3908b570-fcb6-46cb-b28e-0f9a996b0904 None admin] PATH_INFO: `/v3/roles` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:29.451305 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3908b570-fcb6-46cb-b28e-0f9a996b0904 None admin] RBAC: Adding query filter params (name=service) {{(pid=80540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:01:29.451595 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3908b570-fcb6-46cb-b28e-0f9a996b0904 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:01:29.453026 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3908b570-fcb6-46cb-b28e-0f9a996b0904 None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:01:29.460737 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 611/1209] 10.222.0.23 () {56 vars in 1103 bytes} [Thu May 19 12:01:29 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:01:29.466843 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf90af4a-9878-4a34-a703-e3cbb6d5999c None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:01:29.517659 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:01:29.566912 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf90af4a-9878-4a34-a703-e3cbb6d5999c None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:01:29.600102 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8aad39b-1ba1-446d-a769-98df8aa00c91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:01:29.600888 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8aad39b-1ba1-446d-a769-98df8aa00c91 None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:29.601171 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8aad39b-1ba1-446d-a769-98df8aa00c91 None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:29.601446 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8aad39b-1ba1-446d-a769-98df8aa00c91 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:29.612294 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8aad39b-1ba1-446d-a769-98df8aa00c91 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:01:29.613726 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8aad39b-1ba1-446d-a769-98df8aa00c91 None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:01:29.630612 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application [None req-f8aad39b-1ba1-446d-a769-98df8aa00c91 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 12:01:29.630612 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:01:29.630612 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 12:01:29.630612 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 12:01:29.630612 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 12:01:29.630612 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 12:01:29.630612 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 12:01:29.630612 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 12:01:29.630612 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 12:01:29.630612 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 12:01:29.630612 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 12:01:29.630612 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 12:01:29.630612 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 12:01:29.630612 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 12:01:29.630612 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 12:01:29.631566 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 12:01:29.631566 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 12:01:29.631566 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 12:01:29.631566 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 12:01:29.631566 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 12:01:29.631566 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 12:01:29.631566 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.get_or_create( May 19 12:01:29.631566 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 12:01:29.631566 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application with Lock( May 19 12:01:29.631566 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 12:01:29.631566 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._enter() May 19 12:01:29.631566 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 12:01:29.631566 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 12:01:29.631566 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 12:01:29.631566 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.creator() May 19 12:01:29.631566 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 12:01:29.631566 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application created_value = creator( May 19 12:01:29.632540 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 12:01:29.632540 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 12:01:29.632540 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 12:01:29.632540 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application May 19 12:01:29.635946 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 599/1210] 10.222.0.23 () {56 vars in 1108 bytes} [Thu May 19 12:01:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 12:01:29.637823 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3908b570-fcb6-46cb-b28e-0f9a996b0904 None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:01:29.740167 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:01:29.743072 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3908b570-fcb6-46cb-b28e-0f9a996b0904 None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:01:29.776352 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71f8bac-faad-455a-9e43-bab8e71b1737 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:01:29.776930 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71f8bac-faad-455a-9e43-bab8e71b1737 None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:29.777213 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71f8bac-faad-455a-9e43-bab8e71b1737 None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:29.777489 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71f8bac-faad-455a-9e43-bab8e71b1737 None admin] PATH_INFO: `/v3/domains` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:29.778237 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71f8bac-faad-455a-9e43-bab8e71b1737 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:01:29.778520 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71f8bac-faad-455a-9e43-bab8e71b1737 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:01:29.780846 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71f8bac-faad-455a-9e43-bab8e71b1737 None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:01:29.792229 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 612/1211] 10.222.0.23 () {56 vars in 1109 bytes} [Thu May 19 12:01:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:01:29.798119 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8aad39b-1ba1-446d-a769-98df8aa00c91 None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:01:29.902400 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8aad39b-1ba1-446d-a769-98df8aa00c91 None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:01:29.904276 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:01:29.936030 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55c070a-f8c8-442d-851f-ccf1d87fe9ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:01:29.936837 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55c070a-f8c8-442d-851f-ccf1d87fe9ea None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:29.937125 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55c070a-f8c8-442d-851f-ccf1d87fe9ea None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:29.937400 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55c070a-f8c8-442d-851f-ccf1d87fe9ea None admin] PATH_INFO: `/v3/users/placement` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:29.945917 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55c070a-f8c8-442d-851f-ccf1d87fe9ea None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:01:29.947372 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55c070a-f8c8-442d-851f-ccf1d87fe9ea None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:01:29.962564 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application [None req-d55c070a-f8c8-442d-851f-ccf1d87fe9ea None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 19 12:01:29.962564 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:01:29.962564 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 12:01:29.962564 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 12:01:29.962564 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 12:01:29.962564 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 12:01:29.962564 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 12:01:29.962564 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 12:01:29.962564 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 12:01:29.962564 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 12:01:29.962564 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 12:01:29.962564 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 12:01:29.962564 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 12:01:29.962564 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 12:01:29.962564 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 12:01:29.962564 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 12:01:29.963751 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 12:01:29.963751 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 12:01:29.963751 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 12:01:29.963751 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 12:01:29.963751 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 12:01:29.963751 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 12:01:29.963751 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 12:01:29.963751 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 12:01:29.963751 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 12:01:29.963751 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.get_or_create( May 19 12:01:29.963751 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 12:01:29.963751 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application with Lock( May 19 12:01:29.963751 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 12:01:29.963751 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._enter() May 19 12:01:29.963751 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 12:01:29.963751 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 12:01:29.963751 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 12:01:29.963751 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.creator() May 19 12:01:29.964943 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 12:01:29.964943 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application created_value = creator( May 19 12:01:29.964943 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 12:01:29.964943 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 12:01:29.964943 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 12:01:29.964943 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 12:01:29.964943 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 12:01:29.964943 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 12:01:29.964943 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 19 12:01:29.964943 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application May 19 12:01:29.964943 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 600/1212] 10.222.0.23 () {56 vars in 1108 bytes} [Thu May 19 12:01:29 2022] GET /identity/v3/users/placement => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 12:01:29.969669 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71f8bac-faad-455a-9e43-bab8e71b1737 None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:01:30.070153 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71f8bac-faad-455a-9e43-bab8e71b1737 None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:01:30.104243 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8760871-181b-4f78-bcf1-bb139c81f105 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:01:30.104936 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8760871-181b-4f78-bcf1-bb139c81f105 None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:30.105202 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8760871-181b-4f78-bcf1-bb139c81f105 None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:30.105483 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8760871-181b-4f78-bcf1-bb139c81f105 None admin] PATH_INFO: `/v3/users` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:30.106315 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8760871-181b-4f78-bcf1-bb139c81f105 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:01:30.106634 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8760871-181b-4f78-bcf1-bb139c81f105 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:01:30.108402 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8760871-181b-4f78-bcf1-bb139c81f105 None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:01:30.126043 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 613/1213] 10.222.0.23 () {56 vars in 1143 bytes} [Thu May 19 12:01:29 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:01:30.131715 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55c070a-f8c8-442d-851f-ccf1d87fe9ea None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:01:30.235651 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55c070a-f8c8-442d-851f-ccf1d87fe9ea None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:01:30.270326 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f514241-a325-45ce-81a2-9737d45df6fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:01:30.270976 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f514241-a325-45ce-81a2-9737d45df6fd None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:30.271259 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f514241-a325-45ce-81a2-9737d45df6fd None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:30.271537 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f514241-a325-45ce-81a2-9737d45df6fd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:30.301718 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:01:30.339512 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f514241-a325-45ce-81a2-9737d45df6fd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:01:30.341000 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f514241-a325-45ce-81a2-9737d45df6fd None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:01:30.442601 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 601/1214] 10.222.0.23 () {58 vars in 1282 bytes} [Thu May 19 12:01:30 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 12:01:30.448395 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8760871-181b-4f78-bcf1-bb139c81f105 None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:01:30.467463 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:01:30.550160 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8760871-181b-4f78-bcf1-bb139c81f105 None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:01:30.587748 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8ba4a5-f025-40f0-a104-62c66fbf9202 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:01:30.588578 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8ba4a5-f025-40f0-a104-62c66fbf9202 None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:30.588861 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8ba4a5-f025-40f0-a104-62c66fbf9202 None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:30.589136 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8ba4a5-f025-40f0-a104-62c66fbf9202 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:30.601348 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8ba4a5-f025-40f0-a104-62c66fbf9202 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:01:30.602766 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8ba4a5-f025-40f0-a104-62c66fbf9202 None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:01:30.620739 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application [None req-ca8ba4a5-f025-40f0-a104-62c66fbf9202 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 12:01:30.620739 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:01:30.620739 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 12:01:30.620739 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 12:01:30.620739 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 12:01:30.620739 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 12:01:30.620739 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 12:01:30.620739 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 12:01:30.620739 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 12:01:30.620739 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 12:01:30.620739 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 12:01:30.620739 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 12:01:30.620739 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 12:01:30.620739 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 12:01:30.620739 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 12:01:30.621739 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 12:01:30.621739 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 12:01:30.621739 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 12:01:30.621739 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 12:01:30.621739 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 12:01:30.621739 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 12:01:30.621739 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.get_or_create( May 19 12:01:30.621739 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 12:01:30.621739 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application with Lock( May 19 12:01:30.621739 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 12:01:30.621739 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._enter() May 19 12:01:30.621739 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 12:01:30.621739 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 12:01:30.621739 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 12:01:30.621739 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.creator() May 19 12:01:30.621739 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 12:01:30.621739 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application created_value = creator( May 19 12:01:30.623497 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 12:01:30.623497 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 12:01:30.623497 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 12:01:30.623497 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application May 19 12:01:30.623497 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 614/1215] 10.222.0.23 () {56 vars in 1108 bytes} [Thu May 19 12:01:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 12:01:30.628488 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f514241-a325-45ce-81a2-9737d45df6fd None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:01:30.693026 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:01:30.733295 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f514241-a325-45ce-81a2-9737d45df6fd None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:01:30.772199 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446ac29c-9d38-4d8a-bc02-6f1893533691 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:01:30.772859 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446ac29c-9d38-4d8a-bc02-6f1893533691 None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:30.773149 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446ac29c-9d38-4d8a-bc02-6f1893533691 None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:30.773423 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446ac29c-9d38-4d8a-bc02-6f1893533691 None admin] PATH_INFO: `/v3/domains` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:30.774145 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446ac29c-9d38-4d8a-bc02-6f1893533691 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:01:30.774454 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446ac29c-9d38-4d8a-bc02-6f1893533691 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:01:30.775845 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446ac29c-9d38-4d8a-bc02-6f1893533691 None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:01:30.787593 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 602/1216] 10.222.0.23 () {56 vars in 1109 bytes} [Thu May 19 12:01:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 12:01:30.793291 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8ba4a5-f025-40f0-a104-62c66fbf9202 None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:01:30.864108 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:01:30.894300 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8ba4a5-f025-40f0-a104-62c66fbf9202 None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:01:30.928430 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355d3a6b-432e-45ec-bbd7-943e6dc4ecc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:01:30.929236 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355d3a6b-432e-45ec-bbd7-943e6dc4ecc9 None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:30.929513 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355d3a6b-432e-45ec-bbd7-943e6dc4ecc9 None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:30.929794 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355d3a6b-432e-45ec-bbd7-943e6dc4ecc9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:30.936784 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355d3a6b-432e-45ec-bbd7-943e6dc4ecc9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:01:30.938240 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355d3a6b-432e-45ec-bbd7-943e6dc4ecc9 None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:01:30.952100 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application [None req-355d3a6b-432e-45ec-bbd7-943e6dc4ecc9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 12:01:30.952100 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:01:30.952100 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 12:01:30.952100 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 12:01:30.952100 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 12:01:30.952100 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 12:01:30.952100 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 12:01:30.952100 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 12:01:30.952100 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 12:01:30.952100 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 12:01:30.952100 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 12:01:30.952100 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 12:01:30.952100 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 12:01:30.952100 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 12:01:30.952100 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 12:01:30.953226 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 12:01:30.953226 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 12:01:30.953226 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 12:01:30.953226 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 12:01:30.953226 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 12:01:30.953226 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 12:01:30.953226 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.get_or_create( May 19 12:01:30.953226 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 12:01:30.953226 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application with Lock( May 19 12:01:30.953226 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 12:01:30.953226 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._enter() May 19 12:01:30.953226 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 12:01:30.953226 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 12:01:30.953226 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 12:01:30.953226 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.creator() May 19 12:01:30.953226 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 12:01:30.953226 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application created_value = creator( May 19 12:01:30.954360 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 12:01:30.954360 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 12:01:30.954360 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 12:01:30.954360 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 12:01:30.954360 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 12:01:30.954360 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 12:01:30.954360 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 12:01:30.954360 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application May 19 12:01:30.954360 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 615/1217] 10.222.0.23 () {56 vars in 1111 bytes} [Thu May 19 12:01:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 12:01:30.960950 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446ac29c-9d38-4d8a-bc02-6f1893533691 None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:01:31.061826 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446ac29c-9d38-4d8a-bc02-6f1893533691 None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:01:31.084188 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:01:31.096067 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39a3da4-0d22-4546-bd5b-782282cb3cb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:01:31.096762 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39a3da4-0d22-4546-bd5b-782282cb3cb9 None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:31.097041 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39a3da4-0d22-4546-bd5b-782282cb3cb9 None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:31.097315 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39a3da4-0d22-4546-bd5b-782282cb3cb9 None admin] PATH_INFO: `/v3/projects` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:31.098124 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e39a3da4-0d22-4546-bd5b-782282cb3cb9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:01:31.098430 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e39a3da4-0d22-4546-bd5b-782282cb3cb9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:01:31.099879 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e39a3da4-0d22-4546-bd5b-782282cb3cb9 None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:01:31.111646 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 603/1218] 10.222.0.23 () {56 vars in 1148 bytes} [Thu May 19 12:01:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:01:31.117817 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355d3a6b-432e-45ec-bbd7-943e6dc4ecc9 None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:01:31.218798 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355d3a6b-432e-45ec-bbd7-943e6dc4ecc9 None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:01:31.246451 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:01:31.253294 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6d0887-25c3-4b7b-aa86-1db29576a97a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:01:31.253956 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6d0887-25c3-4b7b-aa86-1db29576a97a None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:31.254246 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6d0887-25c3-4b7b-aa86-1db29576a97a None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:31.254527 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6d0887-25c3-4b7b-aa86-1db29576a97a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:31.255390 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6d0887-25c3-4b7b-aa86-1db29576a97a None admin] RBAC: Adding query filter params (role.id=7c59061ba7064a0db6032433fb0eec6b, scope.project.id=d70d25bf69314820b8aaecb65576978a, user.id=50f937eed4f3414fa6f19cab2a84eb5f) {{(pid=80540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:01:31.255710 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6d0887-25c3-4b7b-aa86-1db29576a97a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:01:31.257198 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6d0887-25c3-4b7b-aa86-1db29576a97a None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:01:31.262644 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 616/1219] 10.222.0.23 () {56 vars in 1374 bytes} [Thu May 19 12:01:31 2022] GET /identity/v3/role_assignments?user.id=50f937eed4f3414fa6f19cab2a84eb5f&scope.project.id=d70d25bf69314820b8aaecb65576978a&role.id=7c59061ba7064a0db6032433fb0eec6b => generated 259 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:01:33.490821 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784dfbd2-bc47-42fe-a02a-e6417b3aea50 None None] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:33.491386 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784dfbd2-bc47-42fe-a02a-e6417b3aea50 None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:33.491489 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784dfbd2-bc47-42fe-a02a-e6417b3aea50 None None] PATH_INFO: `/` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:33.492853 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 604/1220] 10.222.0.23 () {54 vars in 926 bytes} [Thu May 19 12:01:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 12:01:33.498838 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4b3442-9b32-401a-bde0-04b0ab12e860 None None] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:33.499247 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4b3442-9b32-401a-bde0-04b0ab12e860 None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:33.499393 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4b3442-9b32-401a-bde0-04b0ab12e860 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:33.516094 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.auth.core [None req-9f4b3442-9b32-401a-bde0-04b0ab12e860 None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:01:33.517276 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-9f4b3442-9b32-401a-bde0-04b0ab12e860 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:01:33.585683 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 617/1221] 10.222.0.23 () {58 vars in 1023 bytes} [Thu May 19 12:01:33 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 12:01:33.608960 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784dfbd2-bc47-42fe-a02a-e6417b3aea50 None None] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:01:33.712882 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784dfbd2-bc47-42fe-a02a-e6417b3aea50 None None] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:01:33.748487 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7fe1d8e-8bce-48df-ae92-874909b8f2f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:01:33.749245 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fe1d8e-8bce-48df-ae92-874909b8f2f2 None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:33.749535 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fe1d8e-8bce-48df-ae92-874909b8f2f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:33.749817 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fe1d8e-8bce-48df-ae92-874909b8f2f2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:33.760053 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7fe1d8e-8bce-48df-ae92-874909b8f2f2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 12:01:33.768040 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application [None req-c7fe1d8e-8bce-48df-ae92-874909b8f2f2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 12:01:33.768040 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:01:33.768040 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 12:01:33.768040 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 12:01:33.768040 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 12:01:33.768040 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 12:01:33.768040 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 12:01:33.768040 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 12:01:33.768040 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 12:01:33.768040 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 12:01:33.768040 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 12:01:33.768040 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 12:01:33.768040 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 12:01:33.768040 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 12:01:33.768040 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 12:01:33.768040 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 12:01:33.769147 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 12:01:33.769147 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 12:01:33.769147 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 12:01:33.769147 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 12:01:33.769147 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 12:01:33.769147 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 12:01:33.769147 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 12:01:33.769147 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 12:01:33.769147 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 12:01:33.769147 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.get_or_create( May 19 12:01:33.769147 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 12:01:33.769147 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application with Lock( May 19 12:01:33.769147 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 12:01:33.769147 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._enter() May 19 12:01:33.769147 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 12:01:33.769147 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 12:01:33.770047 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 12:01:33.770047 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.creator() May 19 12:01:33.770047 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 12:01:33.770047 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application created_value = creator( May 19 12:01:33.770047 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 12:01:33.770047 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 12:01:33.770047 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 12:01:33.770047 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 12:01:33.770047 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 12:01:33.770047 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 12:01:33.770047 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 12:01:33.770047 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application May 19 12:01:33.770047 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 605/1222] 10.222.0.23 () {56 vars in 1102 bytes} [Thu May 19 12:01:33 2022] GET /identity/v3/roles/service => generated 85 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 12:01:33.775234 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4b3442-9b32-401a-bde0-04b0ab12e860 None None] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:01:33.873348 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4b3442-9b32-401a-bde0-04b0ab12e860 None None] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:01:33.895975 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:01:33.906845 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309d84b8-bcb9-4c98-b555-1ee46fb8d14a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:01:33.907512 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309d84b8-bcb9-4c98-b555-1ee46fb8d14a None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:33.907820 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309d84b8-bcb9-4c98-b555-1ee46fb8d14a None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:33.908105 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309d84b8-bcb9-4c98-b555-1ee46fb8d14a None admin] PATH_INFO: `/v3/roles` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:33.908848 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309d84b8-bcb9-4c98-b555-1ee46fb8d14a None admin] RBAC: Adding query filter params (name=service) {{(pid=80540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:01:33.909154 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309d84b8-bcb9-4c98-b555-1ee46fb8d14a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:01:33.910533 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309d84b8-bcb9-4c98-b555-1ee46fb8d14a None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:01:33.918322 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 618/1223] 10.222.0.23 () {56 vars in 1103 bytes} [Thu May 19 12:01:33 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:01:33.925796 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7fe1d8e-8bce-48df-ae92-874909b8f2f2 None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:01:33.969705 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:01:34.028149 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7fe1d8e-8bce-48df-ae92-874909b8f2f2 None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:01:34.063864 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096aee50-2df6-4bc1-a085-850e5bb0a205 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:01:34.064722 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096aee50-2df6-4bc1-a085-850e5bb0a205 None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:34.065009 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096aee50-2df6-4bc1-a085-850e5bb0a205 None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:34.065284 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096aee50-2df6-4bc1-a085-850e5bb0a205 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:34.077463 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096aee50-2df6-4bc1-a085-850e5bb0a205 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:01:34.078905 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096aee50-2df6-4bc1-a085-850e5bb0a205 None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:01:34.097897 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application [None req-096aee50-2df6-4bc1-a085-850e5bb0a205 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 12:01:34.097897 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:01:34.097897 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 12:01:34.097897 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 12:01:34.097897 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 12:01:34.097897 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 12:01:34.097897 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 12:01:34.097897 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 12:01:34.097897 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 12:01:34.097897 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 12:01:34.097897 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 12:01:34.097897 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 12:01:34.097897 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 12:01:34.097897 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 12:01:34.097897 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 12:01:34.099067 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 12:01:34.099067 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 12:01:34.099067 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 12:01:34.099067 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 12:01:34.099067 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 12:01:34.099067 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 12:01:34.099067 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.get_or_create( May 19 12:01:34.099067 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 12:01:34.099067 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application with Lock( May 19 12:01:34.099067 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 12:01:34.099067 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._enter() May 19 12:01:34.099067 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 12:01:34.099067 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 12:01:34.099067 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 12:01:34.099067 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.creator() May 19 12:01:34.099067 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 12:01:34.099067 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application created_value = creator( May 19 12:01:34.104149 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 12:01:34.104149 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 12:01:34.104149 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 12:01:34.104149 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application May 19 12:01:34.104149 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 606/1224] 10.222.0.23 () {56 vars in 1108 bytes} [Thu May 19 12:01:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 12:01:34.105216 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309d84b8-bcb9-4c98-b555-1ee46fb8d14a None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:01:34.208952 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309d84b8-bcb9-4c98-b555-1ee46fb8d14a None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:01:34.243546 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3fbacd-cc70-43ca-82d0-b9853d5806a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:01:34.244319 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3fbacd-cc70-43ca-82d0-b9853d5806a5 None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:34.244609 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3fbacd-cc70-43ca-82d0-b9853d5806a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:34.246899 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3fbacd-cc70-43ca-82d0-b9853d5806a5 None admin] PATH_INFO: `/v3/domains` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:34.246899 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3fbacd-cc70-43ca-82d0-b9853d5806a5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:01:34.246899 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3fbacd-cc70-43ca-82d0-b9853d5806a5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:01:34.247303 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3fbacd-cc70-43ca-82d0-b9853d5806a5 None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:01:34.260254 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 619/1225] 10.222.0.23 () {56 vars in 1109 bytes} [Thu May 19 12:01:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:01:34.266736 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096aee50-2df6-4bc1-a085-850e5bb0a205 None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:01:34.360392 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:01:34.367479 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096aee50-2df6-4bc1-a085-850e5bb0a205 None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:01:34.400659 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbd3a52-dd54-4dd9-8fb9-970419d6d5dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:01:34.401464 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbd3a52-dd54-4dd9-8fb9-970419d6d5dd None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:34.401748 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbd3a52-dd54-4dd9-8fb9-970419d6d5dd None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:34.402023 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbd3a52-dd54-4dd9-8fb9-970419d6d5dd None admin] PATH_INFO: `/v3/users/placement` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:34.409523 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cbd3a52-dd54-4dd9-8fb9-970419d6d5dd None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:01:34.410951 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cbd3a52-dd54-4dd9-8fb9-970419d6d5dd None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:01:34.425757 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application [None req-3cbd3a52-dd54-4dd9-8fb9-970419d6d5dd None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 19 12:01:34.425757 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:01:34.425757 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 12:01:34.425757 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 12:01:34.425757 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 12:01:34.425757 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 12:01:34.425757 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 12:01:34.425757 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 12:01:34.425757 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 12:01:34.425757 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 12:01:34.425757 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 12:01:34.425757 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 12:01:34.425757 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 12:01:34.425757 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 12:01:34.425757 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 12:01:34.425757 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 12:01:34.430276 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 12:01:34.430276 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 12:01:34.430276 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 12:01:34.430276 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 12:01:34.430276 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 12:01:34.430276 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 12:01:34.430276 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 12:01:34.430276 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 12:01:34.430276 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 12:01:34.430276 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.get_or_create( May 19 12:01:34.430276 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 12:01:34.430276 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application with Lock( May 19 12:01:34.430276 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 12:01:34.430276 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._enter() May 19 12:01:34.430276 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 12:01:34.430276 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 12:01:34.430276 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 12:01:34.430276 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.creator() May 19 12:01:34.431579 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 12:01:34.431579 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application created_value = creator( May 19 12:01:34.431579 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 12:01:34.431579 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 12:01:34.431579 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 12:01:34.431579 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 12:01:34.431579 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 12:01:34.431579 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 12:01:34.431579 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 19 12:01:34.431579 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application May 19 12:01:34.431579 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 607/1226] 10.222.0.23 () {56 vars in 1108 bytes} [Thu May 19 12:01:34 2022] GET /identity/v3/users/placement => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 12:01:34.432770 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3fbacd-cc70-43ca-82d0-b9853d5806a5 None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:01:34.479679 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:01:34.534219 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3fbacd-cc70-43ca-82d0-b9853d5806a5 None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:01:34.568651 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3bcf4c-3cf3-4d03-a918-08e8836c5419 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:01:34.569321 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3bcf4c-3cf3-4d03-a918-08e8836c5419 None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:34.569612 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3bcf4c-3cf3-4d03-a918-08e8836c5419 None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:34.569893 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3bcf4c-3cf3-4d03-a918-08e8836c5419 None admin] PATH_INFO: `/v3/users` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:34.570729 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3bcf4c-3cf3-4d03-a918-08e8836c5419 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:01:34.571046 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3bcf4c-3cf3-4d03-a918-08e8836c5419 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:01:34.572539 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3bcf4c-3cf3-4d03-a918-08e8836c5419 None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:01:34.590659 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 620/1227] 10.222.0.23 () {56 vars in 1143 bytes} [Thu May 19 12:01:34 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:01:34.596499 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbd3a52-dd54-4dd9-8fb9-970419d6d5dd None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:01:34.697802 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbd3a52-dd54-4dd9-8fb9-970419d6d5dd None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:01:34.731355 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd16e540-5bb6-43e4-b1cb-a5a9ba7604ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:01:34.732363 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd16e540-5bb6-43e4-b1cb-a5a9ba7604ca None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:34.732732 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd16e540-5bb6-43e4-b1cb-a5a9ba7604ca None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:34.733016 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd16e540-5bb6-43e4-b1cb-a5a9ba7604ca None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:34.743521 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd16e540-5bb6-43e4-b1cb-a5a9ba7604ca None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:01:34.745009 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd16e540-5bb6-43e4-b1cb-a5a9ba7604ca None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:01:34.763722 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application [None req-bd16e540-5bb6-43e4-b1cb-a5a9ba7604ca None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 12:01:34.763722 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:01:34.763722 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 12:01:34.763722 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 12:01:34.763722 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 12:01:34.763722 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 12:01:34.763722 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 12:01:34.763722 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 12:01:34.763722 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 12:01:34.763722 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 12:01:34.763722 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 12:01:34.763722 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 12:01:34.763722 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 12:01:34.763722 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 12:01:34.763722 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 12:01:34.765562 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 12:01:34.765562 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 12:01:34.765562 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 12:01:34.765562 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 12:01:34.765562 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 12:01:34.765562 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 12:01:34.765562 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.get_or_create( May 19 12:01:34.765562 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 12:01:34.765562 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application with Lock( May 19 12:01:34.765562 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 12:01:34.765562 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._enter() May 19 12:01:34.765562 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 12:01:34.765562 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 12:01:34.765562 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 12:01:34.765562 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.creator() May 19 12:01:34.765562 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 12:01:34.765562 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application created_value = creator( May 19 12:01:34.767128 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 12:01:34.767128 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 12:01:34.767128 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 12:01:34.767128 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application May 19 12:01:34.767128 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 608/1228] 10.222.0.23 () {56 vars in 1108 bytes} [Thu May 19 12:01:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 12:01:34.770750 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3bcf4c-3cf3-4d03-a918-08e8836c5419 None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:01:34.870567 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3bcf4c-3cf3-4d03-a918-08e8836c5419 None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:01:34.873678 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:01:34.905322 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc29d24-4a72-4e3b-bf64-a8f057081e4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:01:34.906765 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc29d24-4a72-4e3b-bf64-a8f057081e4f None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:34.907052 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc29d24-4a72-4e3b-bf64-a8f057081e4f None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:34.907328 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc29d24-4a72-4e3b-bf64-a8f057081e4f None admin] PATH_INFO: `/v3/domains` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:34.908132 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc29d24-4a72-4e3b-bf64-a8f057081e4f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:01:34.908439 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc29d24-4a72-4e3b-bf64-a8f057081e4f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:01:34.909816 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc29d24-4a72-4e3b-bf64-a8f057081e4f None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:01:34.921449 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 621/1229] 10.222.0.23 () {56 vars in 1109 bytes} [Thu May 19 12:01:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:01:34.927258 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd16e540-5bb6-43e4-b1cb-a5a9ba7604ca None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:01:34.937955 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:01:35.027944 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd16e540-5bb6-43e4-b1cb-a5a9ba7604ca None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:01:35.062241 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6a4bb1-ebaf-4da0-9c2e-cf972f0c5b34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:01:35.063173 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6a4bb1-ebaf-4da0-9c2e-cf972f0c5b34 None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:35.063461 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6a4bb1-ebaf-4da0-9c2e-cf972f0c5b34 None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:35.063744 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6a4bb1-ebaf-4da0-9c2e-cf972f0c5b34 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:35.070419 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6a4bb1-ebaf-4da0-9c2e-cf972f0c5b34 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:01:35.072185 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6a4bb1-ebaf-4da0-9c2e-cf972f0c5b34 None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:01:35.087458 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application [None req-5c6a4bb1-ebaf-4da0-9c2e-cf972f0c5b34 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 12:01:35.087458 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:01:35.087458 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 12:01:35.087458 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 12:01:35.087458 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 12:01:35.087458 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 12:01:35.087458 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 12:01:35.087458 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 12:01:35.087458 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 12:01:35.087458 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 12:01:35.087458 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 12:01:35.087458 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 12:01:35.087458 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 12:01:35.087458 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 12:01:35.087458 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 12:01:35.088587 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 12:01:35.088587 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 12:01:35.088587 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 12:01:35.088587 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 12:01:35.088587 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 12:01:35.088587 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 12:01:35.088587 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.get_or_create( May 19 12:01:35.088587 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 12:01:35.088587 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application with Lock( May 19 12:01:35.088587 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 12:01:35.088587 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._enter() May 19 12:01:35.088587 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 12:01:35.088587 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 12:01:35.088587 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 12:01:35.088587 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.creator() May 19 12:01:35.088587 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 12:01:35.088587 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application created_value = creator( May 19 12:01:35.089666 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 12:01:35.089666 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 12:01:35.089666 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 12:01:35.089666 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 12:01:35.089666 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 12:01:35.089666 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 12:01:35.089666 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 12:01:35.089666 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application May 19 12:01:35.089666 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 609/1230] 10.222.0.23 () {56 vars in 1111 bytes} [Thu May 19 12:01:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 12:01:35.094734 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc29d24-4a72-4e3b-bf64-a8f057081e4f None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:01:35.195808 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc29d24-4a72-4e3b-bf64-a8f057081e4f None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:01:35.230144 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1721c83-45cd-4d31-bd39-2ad60cc83568 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:01:35.230729 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1721c83-45cd-4d31-bd39-2ad60cc83568 None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:35.231014 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1721c83-45cd-4d31-bd39-2ad60cc83568 None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:35.231292 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1721c83-45cd-4d31-bd39-2ad60cc83568 None admin] PATH_INFO: `/v3/projects` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:35.232702 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1721c83-45cd-4d31-bd39-2ad60cc83568 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:01:35.233181 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1721c83-45cd-4d31-bd39-2ad60cc83568 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:01:35.234618 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1721c83-45cd-4d31-bd39-2ad60cc83568 None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:01:35.246695 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 622/1231] 10.222.0.23 () {56 vars in 1148 bytes} [Thu May 19 12:01:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:01:35.252760 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6a4bb1-ebaf-4da0-9c2e-cf972f0c5b34 None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:01:35.317694 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:01:35.351832 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6a4bb1-ebaf-4da0-9c2e-cf972f0c5b34 None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:01:35.385324 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21108817-b8cb-4dd8-ade8-20217bf16871 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:01:35.386264 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21108817-b8cb-4dd8-ade8-20217bf16871 None admin] REQUEST_METHOD: `PUT` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:35.386543 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21108817-b8cb-4dd8-ade8-20217bf16871 None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:35.386815 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21108817-b8cb-4dd8-ade8-20217bf16871 None admin] PATH_INFO: `/v3/projects/d70d25bf69314820b8aaecb65576978a/users/50f937eed4f3414fa6f19cab2a84eb5f/roles/7c59061ba7064a0db6032433fb0eec6b` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:35.401560 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21108817-b8cb-4dd8-ade8-20217bf16871 None admin] RBAC: Authorizing `identity:create_grant(project_id=d70d25bf69314820b8aaecb65576978a, user_id=50f937eed4f3414fa6f19cab2a84eb5f, role_id=7c59061ba7064a0db6032433fb0eec6b)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:01:35.403000 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21108817-b8cb-4dd8-ade8-20217bf16871 None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:01:35.474771 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 610/1232] 10.222.0.23 () {58 vars in 1439 bytes} [Thu May 19 12:01:35 2022] PUT /identity/v3/projects/d70d25bf69314820b8aaecb65576978a/users/50f937eed4f3414fa6f19cab2a84eb5f/roles/7c59061ba7064a0db6032433fb0eec6b => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:01:37.657615 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adb580b-ff08-46bc-86df-f4ce58f65e2b None None] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:37.658142 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adb580b-ff08-46bc-86df-f4ce58f65e2b None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:37.658251 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adb580b-ff08-46bc-86df-f4ce58f65e2b None None] PATH_INFO: `/` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:37.659696 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 623/1233] 10.222.0.23 () {54 vars in 926 bytes} [Thu May 19 12:01:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 12:01:37.665035 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30daeda-e7ea-4e91-99b5-be4afcf9e073 None None] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:37.665465 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30daeda-e7ea-4e91-99b5-be4afcf9e073 None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:37.665576 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30daeda-e7ea-4e91-99b5-be4afcf9e073 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:37.682262 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.auth.core [None req-b30daeda-e7ea-4e91-99b5-be4afcf9e073 None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:01:37.683453 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-b30daeda-e7ea-4e91-99b5-be4afcf9e073 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:01:37.751042 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 611/1234] 10.222.0.23 () {58 vars in 1023 bytes} [Thu May 19 12:01:37 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 12:01:37.775316 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0adb580b-ff08-46bc-86df-f4ce58f65e2b None None] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:01:37.802915 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:01:37.880384 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0adb580b-ff08-46bc-86df-f4ce58f65e2b None None] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:01:37.915021 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ce5b09-8c44-4a45-9f80-61471d3bf129 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:01:37.915750 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ce5b09-8c44-4a45-9f80-61471d3bf129 None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:37.916059 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ce5b09-8c44-4a45-9f80-61471d3bf129 None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:37.916338 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ce5b09-8c44-4a45-9f80-61471d3bf129 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:37.929295 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ce5b09-8c44-4a45-9f80-61471d3bf129 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 12:01:37.934432 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application [None req-54ce5b09-8c44-4a45-9f80-61471d3bf129 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 12:01:37.934432 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:01:37.934432 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 12:01:37.934432 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 12:01:37.934432 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 12:01:37.934432 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 12:01:37.934432 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 12:01:37.934432 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 12:01:37.934432 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 12:01:37.934432 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 12:01:37.934432 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 12:01:37.934432 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 12:01:37.934432 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 12:01:37.934432 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 12:01:37.934432 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 12:01:37.934432 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 12:01:37.935518 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 12:01:37.935518 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 12:01:37.935518 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 12:01:37.935518 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 12:01:37.935518 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 12:01:37.935518 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 12:01:37.935518 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 12:01:37.935518 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 12:01:37.935518 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 12:01:37.935518 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.get_or_create( May 19 12:01:37.935518 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 12:01:37.935518 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application with Lock( May 19 12:01:37.935518 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 12:01:37.935518 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._enter() May 19 12:01:37.935518 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 12:01:37.935518 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 12:01:37.940594 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 12:01:37.940594 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.creator() May 19 12:01:37.940594 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 12:01:37.940594 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application created_value = creator( May 19 12:01:37.940594 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 12:01:37.940594 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 12:01:37.940594 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 12:01:37.940594 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 12:01:37.940594 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 12:01:37.940594 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 12:01:37.940594 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 12:01:37.940594 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application May 19 12:01:37.940594 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 624/1235] 10.222.0.23 () {56 vars in 1102 bytes} [Thu May 19 12:01:37 2022] GET /identity/v3/roles/service => generated 85 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 12:01:37.941921 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30daeda-e7ea-4e91-99b5-be4afcf9e073 None None] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:01:37.988206 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:01:38.040505 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30daeda-e7ea-4e91-99b5-be4afcf9e073 None None] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:01:38.073560 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12c3b17-c8de-4383-bdb4-17c3adffe88a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:01:38.074255 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12c3b17-c8de-4383-bdb4-17c3adffe88a None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:38.074551 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12c3b17-c8de-4383-bdb4-17c3adffe88a None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:38.074837 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12c3b17-c8de-4383-bdb4-17c3adffe88a None admin] PATH_INFO: `/v3/roles` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:38.076378 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12c3b17-c8de-4383-bdb4-17c3adffe88a None admin] RBAC: Adding query filter params (name=service) {{(pid=80539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:01:38.076690 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12c3b17-c8de-4383-bdb4-17c3adffe88a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:01:38.078065 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12c3b17-c8de-4383-bdb4-17c3adffe88a None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:01:38.086806 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 612/1236] 10.222.0.23 () {56 vars in 1103 bytes} [Thu May 19 12:01:37 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 12:01:38.093942 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ce5b09-8c44-4a45-9f80-61471d3bf129 None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:01:38.187702 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:01:38.204071 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ce5b09-8c44-4a45-9f80-61471d3bf129 None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:01:38.237515 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a8e6fc-48b4-4c58-8b01-090d7a39ad2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:01:38.238301 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a8e6fc-48b4-4c58-8b01-090d7a39ad2f None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:38.238584 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a8e6fc-48b4-4c58-8b01-090d7a39ad2f None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:38.238858 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a8e6fc-48b4-4c58-8b01-090d7a39ad2f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:38.249483 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a8e6fc-48b4-4c58-8b01-090d7a39ad2f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:01:38.250954 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a8e6fc-48b4-4c58-8b01-090d7a39ad2f None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:01:38.269187 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application [None req-29a8e6fc-48b4-4c58-8b01-090d7a39ad2f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 12:01:38.269187 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:01:38.269187 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 12:01:38.269187 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 12:01:38.269187 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 12:01:38.269187 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 12:01:38.269187 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 12:01:38.269187 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 12:01:38.269187 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 12:01:38.269187 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 12:01:38.269187 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 12:01:38.269187 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 12:01:38.269187 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 12:01:38.269187 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 12:01:38.269187 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 12:01:38.270136 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 12:01:38.270136 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 12:01:38.270136 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 12:01:38.270136 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 12:01:38.270136 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 12:01:38.270136 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 12:01:38.270136 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.get_or_create( May 19 12:01:38.270136 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 12:01:38.270136 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application with Lock( May 19 12:01:38.270136 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 12:01:38.270136 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._enter() May 19 12:01:38.270136 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 12:01:38.270136 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 12:01:38.270136 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 12:01:38.270136 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.creator() May 19 12:01:38.270136 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 12:01:38.270136 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application created_value = creator( May 19 12:01:38.271060 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 12:01:38.271060 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 12:01:38.271060 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 12:01:38.271060 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application May 19 12:01:38.271060 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 625/1237] 10.222.0.23 () {56 vars in 1108 bytes} [Thu May 19 12:01:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 12:01:38.276605 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12c3b17-c8de-4383-bdb4-17c3adffe88a None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:01:38.379078 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12c3b17-c8de-4383-bdb4-17c3adffe88a None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:01:38.388778 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:01:38.413442 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931cd241-be3c-4105-870d-a6225e38cbc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:01:38.414104 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931cd241-be3c-4105-870d-a6225e38cbc0 None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:38.414394 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931cd241-be3c-4105-870d-a6225e38cbc0 None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:38.414665 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931cd241-be3c-4105-870d-a6225e38cbc0 None admin] PATH_INFO: `/v3/domains` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:38.415388 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-931cd241-be3c-4105-870d-a6225e38cbc0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:01:38.415693 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-931cd241-be3c-4105-870d-a6225e38cbc0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:01:38.417143 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-931cd241-be3c-4105-870d-a6225e38cbc0 None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:01:38.428436 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 613/1238] 10.222.0.23 () {56 vars in 1109 bytes} [Thu May 19 12:01:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:01:38.434097 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a8e6fc-48b4-4c58-8b01-090d7a39ad2f None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:01:38.539354 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a8e6fc-48b4-4c58-8b01-090d7a39ad2f None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:01:38.573658 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d8e20d-0812-4167-b237-1eb2f2bd6d75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:01:38.574454 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d8e20d-0812-4167-b237-1eb2f2bd6d75 None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:38.574740 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d8e20d-0812-4167-b237-1eb2f2bd6d75 None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:38.575019 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d8e20d-0812-4167-b237-1eb2f2bd6d75 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:38.578127 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:01:38.582555 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d8e20d-0812-4167-b237-1eb2f2bd6d75 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:01:38.584033 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d8e20d-0812-4167-b237-1eb2f2bd6d75 None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:01:38.598831 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application [None req-35d8e20d-0812-4167-b237-1eb2f2bd6d75 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 19 12:01:38.598831 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:01:38.598831 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 12:01:38.598831 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 12:01:38.598831 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 12:01:38.598831 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 12:01:38.598831 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 12:01:38.598831 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 12:01:38.598831 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 12:01:38.598831 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 12:01:38.598831 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 12:01:38.598831 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 12:01:38.598831 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 12:01:38.598831 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 12:01:38.598831 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 12:01:38.598831 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 12:01:38.600115 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 12:01:38.600115 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 12:01:38.600115 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 12:01:38.600115 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 12:01:38.600115 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 12:01:38.600115 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 12:01:38.600115 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 12:01:38.600115 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 12:01:38.600115 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 12:01:38.600115 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.get_or_create( May 19 12:01:38.600115 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 12:01:38.600115 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application with Lock( May 19 12:01:38.600115 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 12:01:38.600115 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._enter() May 19 12:01:38.600115 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 12:01:38.600115 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 12:01:38.600115 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 12:01:38.600115 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.creator() May 19 12:01:38.601395 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 12:01:38.601395 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application created_value = creator( May 19 12:01:38.601395 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 12:01:38.601395 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 12:01:38.601395 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 12:01:38.601395 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 12:01:38.601395 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 12:01:38.601395 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 12:01:38.601395 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 19 12:01:38.601395 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application May 19 12:01:38.601395 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 626/1239] 10.222.0.23 () {56 vars in 1108 bytes} [Thu May 19 12:01:38 2022] GET /identity/v3/users/placement => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 12:01:38.605900 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931cd241-be3c-4105-870d-a6225e38cbc0 None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:01:38.704619 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931cd241-be3c-4105-870d-a6225e38cbc0 None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:01:38.738496 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c98aca6-84ea-4e4d-a205-705491e173ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:01:38.739155 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c98aca6-84ea-4e4d-a205-705491e173ca None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:38.739439 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c98aca6-84ea-4e4d-a205-705491e173ca None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:38.739715 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c98aca6-84ea-4e4d-a205-705491e173ca None admin] PATH_INFO: `/v3/users` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:38.740632 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c98aca6-84ea-4e4d-a205-705491e173ca None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:01:38.740877 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c98aca6-84ea-4e4d-a205-705491e173ca None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:01:38.742327 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c98aca6-84ea-4e4d-a205-705491e173ca None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:01:38.762136 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 614/1240] 10.222.0.23 () {56 vars in 1143 bytes} [Thu May 19 12:01:38 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:01:38.768225 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d8e20d-0812-4167-b237-1eb2f2bd6d75 None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:01:38.869338 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d8e20d-0812-4167-b237-1eb2f2bd6d75 None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:01:38.905028 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71272b24-e78e-4f0c-8cab-1239700b8a75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:01:38.905673 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71272b24-e78e-4f0c-8cab-1239700b8a75 None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:38.905962 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71272b24-e78e-4f0c-8cab-1239700b8a75 None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:38.906264 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71272b24-e78e-4f0c-8cab-1239700b8a75 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:38.971656 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:01:38.983155 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71272b24-e78e-4f0c-8cab-1239700b8a75 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:01:38.984688 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71272b24-e78e-4f0c-8cab-1239700b8a75 None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:01:39.080733 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 627/1241] 10.222.0.23 () {58 vars in 1282 bytes} [Thu May 19 12:01:38 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 12:01:39.086597 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c98aca6-84ea-4e4d-a205-705491e173ca None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:01:39.114388 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:01:39.187209 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c98aca6-84ea-4e4d-a205-705491e173ca None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:01:39.221293 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8078c137-01fa-4479-8289-2c1cba5ffe02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:01:39.222085 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8078c137-01fa-4479-8289-2c1cba5ffe02 None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:39.222373 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8078c137-01fa-4479-8289-2c1cba5ffe02 None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:39.222652 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8078c137-01fa-4479-8289-2c1cba5ffe02 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:39.233494 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8078c137-01fa-4479-8289-2c1cba5ffe02 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:01:39.234934 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8078c137-01fa-4479-8289-2c1cba5ffe02 None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:01:39.252252 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application [None req-8078c137-01fa-4479-8289-2c1cba5ffe02 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 12:01:39.252252 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:01:39.252252 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 12:01:39.252252 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 12:01:39.252252 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 12:01:39.252252 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 12:01:39.252252 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 12:01:39.252252 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 12:01:39.252252 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 12:01:39.252252 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 12:01:39.252252 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 12:01:39.252252 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 12:01:39.252252 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 12:01:39.252252 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 12:01:39.252252 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 12:01:39.253381 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 12:01:39.253381 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 12:01:39.253381 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 12:01:39.253381 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 12:01:39.253381 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 12:01:39.253381 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 12:01:39.253381 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.get_or_create( May 19 12:01:39.253381 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 12:01:39.253381 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application with Lock( May 19 12:01:39.253381 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 12:01:39.253381 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._enter() May 19 12:01:39.253381 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 12:01:39.253381 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 12:01:39.253381 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 12:01:39.253381 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.creator() May 19 12:01:39.253381 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 12:01:39.253381 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application created_value = creator( May 19 12:01:39.259149 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 12:01:39.259149 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 12:01:39.259149 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 12:01:39.259149 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application May 19 12:01:39.259149 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 615/1242] 10.222.0.23 () {56 vars in 1108 bytes} [Thu May 19 12:01:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 12:01:39.259526 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71272b24-e78e-4f0c-8cab-1239700b8a75 None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:01:39.357688 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:01:39.363420 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71272b24-e78e-4f0c-8cab-1239700b8a75 None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:01:39.397098 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65db5d84-c601-48ae-ad47-dae892c61e7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:01:39.397778 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65db5d84-c601-48ae-ad47-dae892c61e7d None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:39.398069 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65db5d84-c601-48ae-ad47-dae892c61e7d None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:39.398349 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65db5d84-c601-48ae-ad47-dae892c61e7d None admin] PATH_INFO: `/v3/domains` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:39.399093 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65db5d84-c601-48ae-ad47-dae892c61e7d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:01:39.399400 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65db5d84-c601-48ae-ad47-dae892c61e7d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:01:39.400850 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65db5d84-c601-48ae-ad47-dae892c61e7d None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:01:39.412468 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 628/1243] 10.222.0.23 () {56 vars in 1109 bytes} [Thu May 19 12:01:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:01:39.418332 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8078c137-01fa-4479-8289-2c1cba5ffe02 None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:01:39.496321 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:01:39.517508 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8078c137-01fa-4479-8289-2c1cba5ffe02 None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:01:39.550405 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be8a4a4-4839-484f-9aa1-733f2f5ef11c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:01:39.551187 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be8a4a4-4839-484f-9aa1-733f2f5ef11c None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:39.551472 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be8a4a4-4839-484f-9aa1-733f2f5ef11c None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:39.551749 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be8a4a4-4839-484f-9aa1-733f2f5ef11c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:39.558428 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be8a4a4-4839-484f-9aa1-733f2f5ef11c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:01:39.559934 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be8a4a4-4839-484f-9aa1-733f2f5ef11c None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:01:39.574278 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application [None req-0be8a4a4-4839-484f-9aa1-733f2f5ef11c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 12:01:39.574278 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:01:39.574278 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 12:01:39.574278 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 12:01:39.574278 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 12:01:39.574278 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 12:01:39.574278 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 12:01:39.574278 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 12:01:39.574278 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 12:01:39.574278 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 12:01:39.574278 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 12:01:39.574278 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 12:01:39.574278 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 12:01:39.574278 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 12:01:39.574278 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 12:01:39.575263 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 12:01:39.575263 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 12:01:39.575263 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 12:01:39.575263 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 12:01:39.575263 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 12:01:39.575263 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 12:01:39.575263 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.get_or_create( May 19 12:01:39.575263 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 12:01:39.575263 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application with Lock( May 19 12:01:39.575263 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 12:01:39.575263 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._enter() May 19 12:01:39.575263 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 12:01:39.575263 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 12:01:39.575263 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 12:01:39.575263 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.creator() May 19 12:01:39.575263 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 12:01:39.575263 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application created_value = creator( May 19 12:01:39.577353 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 12:01:39.577353 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 12:01:39.577353 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 12:01:39.577353 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 12:01:39.577353 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 12:01:39.577353 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 12:01:39.577353 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 12:01:39.577353 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application May 19 12:01:39.577353 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 616/1244] 10.222.0.23 () {56 vars in 1111 bytes} [Thu May 19 12:01:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 12:01:39.582528 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65db5d84-c601-48ae-ad47-dae892c61e7d None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:01:39.683441 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65db5d84-c601-48ae-ad47-dae892c61e7d None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:01:39.717993 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6e15f8-fbcb-4f23-aec7-4f9686e46259 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:01:39.719262 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6e15f8-fbcb-4f23-aec7-4f9686e46259 None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:39.719262 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6e15f8-fbcb-4f23-aec7-4f9686e46259 None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:39.719262 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6e15f8-fbcb-4f23-aec7-4f9686e46259 None admin] PATH_INFO: `/v3/projects` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:39.720024 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6e15f8-fbcb-4f23-aec7-4f9686e46259 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:01:39.720334 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6e15f8-fbcb-4f23-aec7-4f9686e46259 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:01:39.721794 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6e15f8-fbcb-4f23-aec7-4f9686e46259 None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:01:39.733942 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 629/1245] 10.222.0.23 () {56 vars in 1148 bytes} [Thu May 19 12:01:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 12:01:39.740050 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be8a4a4-4839-484f-9aa1-733f2f5ef11c None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:01:39.840073 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be8a4a4-4839-484f-9aa1-733f2f5ef11c None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:01:39.873342 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6801ce04-e0aa-471c-ad10-49edf973283e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:01:39.874018 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6801ce04-e0aa-471c-ad10-49edf973283e None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:39.874306 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6801ce04-e0aa-471c-ad10-49edf973283e None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:39.874588 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6801ce04-e0aa-471c-ad10-49edf973283e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:39.875421 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6801ce04-e0aa-471c-ad10-49edf973283e None admin] RBAC: Adding query filter params (role.id=7c59061ba7064a0db6032433fb0eec6b, scope.project.id=d70d25bf69314820b8aaecb65576978a, user.id=50f937eed4f3414fa6f19cab2a84eb5f) {{(pid=80539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:01:39.875740 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6801ce04-e0aa-471c-ad10-49edf973283e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:01:39.877204 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6801ce04-e0aa-471c-ad10-49edf973283e None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:01:39.883026 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 617/1246] 10.222.0.23 () {56 vars in 1374 bytes} [Thu May 19 12:01:39 2022] GET /identity/v3/role_assignments?user.id=50f937eed4f3414fa6f19cab2a84eb5f&scope.project.id=d70d25bf69314820b8aaecb65576978a&role.id=7c59061ba7064a0db6032433fb0eec6b => generated 608 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:01:42.167017 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27922646-75d7-4e44-ad4d-1aedeafc0ce9 None None] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:42.167539 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27922646-75d7-4e44-ad4d-1aedeafc0ce9 None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:42.167632 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27922646-75d7-4e44-ad4d-1aedeafc0ce9 None None] PATH_INFO: `/` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:42.169006 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 630/1247] 10.222.0.23 () {54 vars in 926 bytes} [Thu May 19 12:01:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 12:01:42.174326 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee559584-6999-4520-b5a8-15f50837dca5 None None] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:42.174739 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee559584-6999-4520-b5a8-15f50837dca5 None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:42.174919 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee559584-6999-4520-b5a8-15f50837dca5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:42.179956 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:01:42.193136 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.auth.core [None req-ee559584-6999-4520-b5a8-15f50837dca5 None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:01:42.196011 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-ee559584-6999-4520-b5a8-15f50837dca5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:01:42.262376 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 618/1248] 10.222.0.23 () {58 vars in 1023 bytes} [Thu May 19 12:01:42 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 12:01:42.286275 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27922646-75d7-4e44-ad4d-1aedeafc0ce9 None None] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:01:42.304964 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:01:42.396593 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27922646-75d7-4e44-ad4d-1aedeafc0ce9 None None] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:01:42.433958 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050a6317-4d2a-4441-9013-25ef1d9d9b27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:01:42.435540 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050a6317-4d2a-4441-9013-25ef1d9d9b27 None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:42.446468 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050a6317-4d2a-4441-9013-25ef1d9d9b27 None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:42.446468 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050a6317-4d2a-4441-9013-25ef1d9d9b27 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:42.448168 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050a6317-4d2a-4441-9013-25ef1d9d9b27 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 12:01:42.457549 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application [None req-050a6317-4d2a-4441-9013-25ef1d9d9b27 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 12:01:42.457549 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:01:42.457549 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 12:01:42.457549 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 12:01:42.457549 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 12:01:42.457549 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 12:01:42.457549 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 12:01:42.457549 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 12:01:42.457549 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 12:01:42.457549 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 12:01:42.457549 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 12:01:42.457549 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 12:01:42.457549 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 12:01:42.457549 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 12:01:42.457549 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 12:01:42.457549 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 12:01:42.458807 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 12:01:42.458807 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 12:01:42.458807 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 12:01:42.458807 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 12:01:42.458807 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 12:01:42.458807 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 12:01:42.458807 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 12:01:42.458807 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 12:01:42.458807 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 12:01:42.458807 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.get_or_create( May 19 12:01:42.458807 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 12:01:42.458807 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application with Lock( May 19 12:01:42.458807 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 12:01:42.458807 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._enter() May 19 12:01:42.458807 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 12:01:42.458807 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 12:01:42.459902 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 12:01:42.459902 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.creator() May 19 12:01:42.459902 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 12:01:42.459902 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application created_value = creator( May 19 12:01:42.459902 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 12:01:42.459902 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 12:01:42.459902 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 12:01:42.459902 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 12:01:42.459902 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 12:01:42.459902 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 12:01:42.459902 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 12:01:42.459902 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application May 19 12:01:42.464099 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 631/1249] 10.222.0.23 () {56 vars in 1096 bytes} [Thu May 19 12:01:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 12:01:42.465315 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee559584-6999-4520-b5a8-15f50837dca5 None None] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:01:42.564262 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee559584-6999-4520-b5a8-15f50837dca5 None None] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:01:42.593644 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:01:42.598710 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f9cc06-6a8a-4894-82b3-c451d7070cfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:01:42.599380 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f9cc06-6a8a-4894-82b3-c451d7070cfb None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:42.599668 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f9cc06-6a8a-4894-82b3-c451d7070cfb None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:42.599972 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f9cc06-6a8a-4894-82b3-c451d7070cfb None admin] PATH_INFO: `/v3/roles` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:42.600743 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f9cc06-6a8a-4894-82b3-c451d7070cfb None admin] RBAC: Adding query filter params (name=admin) {{(pid=80539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:01:42.601052 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f9cc06-6a8a-4894-82b3-c451d7070cfb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:01:42.602444 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f9cc06-6a8a-4894-82b3-c451d7070cfb None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:01:42.611503 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 619/1250] 10.222.0.23 () {56 vars in 1099 bytes} [Thu May 19 12:01:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 12:01:42.617295 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050a6317-4d2a-4441-9013-25ef1d9d9b27 None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:01:42.695547 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:01:42.719360 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050a6317-4d2a-4441-9013-25ef1d9d9b27 None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:01:42.753986 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041c3137-d323-4d17-9e72-5bb3c86fa395 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:01:42.754779 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041c3137-d323-4d17-9e72-5bb3c86fa395 None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:42.755065 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041c3137-d323-4d17-9e72-5bb3c86fa395 None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:42.755346 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041c3137-d323-4d17-9e72-5bb3c86fa395 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:42.765942 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041c3137-d323-4d17-9e72-5bb3c86fa395 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:01:42.767417 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041c3137-d323-4d17-9e72-5bb3c86fa395 None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:01:42.784728 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application [None req-041c3137-d323-4d17-9e72-5bb3c86fa395 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 12:01:42.784728 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:01:42.784728 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 12:01:42.784728 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 12:01:42.784728 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 12:01:42.784728 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 12:01:42.784728 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 12:01:42.784728 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 12:01:42.784728 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 12:01:42.784728 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 12:01:42.784728 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 12:01:42.784728 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 12:01:42.784728 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 12:01:42.784728 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 12:01:42.784728 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 12:01:42.785827 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 12:01:42.785827 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 12:01:42.785827 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 12:01:42.785827 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 12:01:42.785827 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 12:01:42.785827 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 12:01:42.785827 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.get_or_create( May 19 12:01:42.785827 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 12:01:42.785827 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application with Lock( May 19 12:01:42.785827 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 12:01:42.785827 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._enter() May 19 12:01:42.785827 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 12:01:42.785827 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 12:01:42.785827 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 12:01:42.785827 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.creator() May 19 12:01:42.785827 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 12:01:42.785827 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application created_value = creator( May 19 12:01:42.786932 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 12:01:42.786932 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 12:01:42.786932 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 12:01:42.786932 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application May 19 12:01:42.786932 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 632/1251] 10.222.0.23 () {56 vars in 1108 bytes} [Thu May 19 12:01:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 12:01:42.791688 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f9cc06-6a8a-4894-82b3-c451d7070cfb None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:01:42.894061 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f9cc06-6a8a-4894-82b3-c451d7070cfb None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:01:42.928643 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4535660e-b0a4-4895-ad1c-4aa9ffff698b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:01:42.929303 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4535660e-b0a4-4895-ad1c-4aa9ffff698b None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:42.929586 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4535660e-b0a4-4895-ad1c-4aa9ffff698b None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:42.929862 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4535660e-b0a4-4895-ad1c-4aa9ffff698b None admin] PATH_INFO: `/v3/domains` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:42.930583 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4535660e-b0a4-4895-ad1c-4aa9ffff698b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:01:42.930885 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4535660e-b0a4-4895-ad1c-4aa9ffff698b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:01:42.932294 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4535660e-b0a4-4895-ad1c-4aa9ffff698b None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:01:42.944659 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 620/1252] 10.222.0.23 () {56 vars in 1109 bytes} [Thu May 19 12:01:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:01:42.950123 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041c3137-d323-4d17-9e72-5bb3c86fa395 None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:01:43.051818 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041c3137-d323-4d17-9e72-5bb3c86fa395 None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:01:43.078277 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:01:43.086215 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57bee68-f8b1-431a-96fe-9d8aeba13e41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:01:43.087015 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57bee68-f8b1-431a-96fe-9d8aeba13e41 None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:43.087300 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57bee68-f8b1-431a-96fe-9d8aeba13e41 None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:43.087575 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57bee68-f8b1-431a-96fe-9d8aeba13e41 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:43.095736 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57bee68-f8b1-431a-96fe-9d8aeba13e41 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:01:43.097425 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57bee68-f8b1-431a-96fe-9d8aeba13e41 None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:01:43.111881 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application [None req-b57bee68-f8b1-431a-96fe-9d8aeba13e41 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 19 12:01:43.111881 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:01:43.111881 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 12:01:43.111881 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 12:01:43.111881 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 12:01:43.111881 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 12:01:43.111881 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 12:01:43.111881 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 12:01:43.111881 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 12:01:43.111881 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 12:01:43.111881 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 12:01:43.111881 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 12:01:43.111881 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 12:01:43.111881 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 12:01:43.111881 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 12:01:43.111881 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 12:01:43.112992 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 12:01:43.112992 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 12:01:43.112992 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 12:01:43.112992 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 12:01:43.112992 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 12:01:43.112992 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 12:01:43.112992 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 12:01:43.112992 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 12:01:43.112992 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 12:01:43.112992 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.get_or_create( May 19 12:01:43.112992 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 12:01:43.112992 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application with Lock( May 19 12:01:43.112992 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 12:01:43.112992 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._enter() May 19 12:01:43.112992 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 12:01:43.112992 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 12:01:43.112992 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 12:01:43.112992 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.creator() May 19 12:01:43.114337 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 12:01:43.114337 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application created_value = creator( May 19 12:01:43.114337 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 12:01:43.114337 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 12:01:43.114337 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 12:01:43.114337 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 12:01:43.114337 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 12:01:43.114337 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 12:01:43.114337 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 19 12:01:43.114337 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application May 19 12:01:43.114337 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 633/1253] 10.222.0.23 () {56 vars in 1108 bytes} [Thu May 19 12:01:42 2022] GET /identity/v3/users/placement => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 12:01:43.118884 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4535660e-b0a4-4895-ad1c-4aa9ffff698b None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:01:43.173705 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:01:43.217949 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4535660e-b0a4-4895-ad1c-4aa9ffff698b None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:01:43.251274 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be96f04b-4ea8-461b-be30-1c7346b2ee31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:01:43.251980 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be96f04b-4ea8-461b-be30-1c7346b2ee31 None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:43.252268 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be96f04b-4ea8-461b-be30-1c7346b2ee31 None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:43.252545 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be96f04b-4ea8-461b-be30-1c7346b2ee31 None admin] PATH_INFO: `/v3/users` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:43.253405 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be96f04b-4ea8-461b-be30-1c7346b2ee31 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:01:43.253717 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be96f04b-4ea8-461b-be30-1c7346b2ee31 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:01:43.255143 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be96f04b-4ea8-461b-be30-1c7346b2ee31 None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:01:43.273224 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 621/1254] 10.222.0.23 () {56 vars in 1143 bytes} [Thu May 19 12:01:43 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:01:43.279850 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57bee68-f8b1-431a-96fe-9d8aeba13e41 None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:01:43.382012 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57bee68-f8b1-431a-96fe-9d8aeba13e41 None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:01:43.415515 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e900d05c-7787-4a27-9601-0a61357517ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:01:43.416250 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e900d05c-7787-4a27-9601-0a61357517ef None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:43.416515 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e900d05c-7787-4a27-9601-0a61357517ef None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:43.416845 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e900d05c-7787-4a27-9601-0a61357517ef None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:43.474360 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:01:43.486275 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e900d05c-7787-4a27-9601-0a61357517ef None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:01:43.487759 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e900d05c-7787-4a27-9601-0a61357517ef None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:01:43.588559 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 634/1255] 10.222.0.23 () {58 vars in 1282 bytes} [Thu May 19 12:01:43 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 12:01:43.594628 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be96f04b-4ea8-461b-be30-1c7346b2ee31 None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:01:43.696079 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be96f04b-4ea8-461b-be30-1c7346b2ee31 None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:01:43.710348 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:01:43.731118 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4873db-23af-428b-8e08-7e1750fd13e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:01:43.731893 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4873db-23af-428b-8e08-7e1750fd13e7 None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:43.732180 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4873db-23af-428b-8e08-7e1750fd13e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:43.732453 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4873db-23af-428b-8e08-7e1750fd13e7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:43.742471 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4873db-23af-428b-8e08-7e1750fd13e7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:01:43.743943 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4873db-23af-428b-8e08-7e1750fd13e7 None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:01:43.761598 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application [None req-6a4873db-23af-428b-8e08-7e1750fd13e7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 12:01:43.761598 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:01:43.761598 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 12:01:43.761598 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 12:01:43.761598 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 12:01:43.761598 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 12:01:43.761598 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 12:01:43.761598 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 12:01:43.761598 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 12:01:43.761598 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 12:01:43.761598 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 12:01:43.761598 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 12:01:43.761598 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 12:01:43.761598 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 12:01:43.761598 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 12:01:43.762678 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 12:01:43.762678 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 12:01:43.762678 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 12:01:43.762678 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 12:01:43.762678 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 12:01:43.762678 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 12:01:43.762678 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.get_or_create( May 19 12:01:43.762678 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 12:01:43.762678 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application with Lock( May 19 12:01:43.762678 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 12:01:43.762678 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._enter() May 19 12:01:43.762678 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 12:01:43.762678 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 12:01:43.762678 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 12:01:43.762678 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.creator() May 19 12:01:43.762678 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 12:01:43.762678 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application created_value = creator( May 19 12:01:43.764427 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 12:01:43.764427 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 12:01:43.764427 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 12:01:43.764427 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application May 19 12:01:43.764427 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 622/1256] 10.222.0.23 () {56 vars in 1108 bytes} [Thu May 19 12:01:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 12:01:43.769532 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e900d05c-7787-4a27-9601-0a61357517ef None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:01:43.855811 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:01:43.867839 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e900d05c-7787-4a27-9601-0a61357517ef None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:01:43.901424 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8e3ed3-baf9-4a63-88eb-20f1e5c58f73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:01:43.902079 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8e3ed3-baf9-4a63-88eb-20f1e5c58f73 None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:43.902362 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8e3ed3-baf9-4a63-88eb-20f1e5c58f73 None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:43.902635 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8e3ed3-baf9-4a63-88eb-20f1e5c58f73 None admin] PATH_INFO: `/v3/domains` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:43.903345 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f8e3ed3-baf9-4a63-88eb-20f1e5c58f73 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:01:43.903652 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f8e3ed3-baf9-4a63-88eb-20f1e5c58f73 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:01:43.905160 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f8e3ed3-baf9-4a63-88eb-20f1e5c58f73 None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:01:43.916865 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 635/1257] 10.222.0.23 () {56 vars in 1109 bytes} [Thu May 19 12:01:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:01:43.922735 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4873db-23af-428b-8e08-7e1750fd13e7 None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:01:44.028545 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4873db-23af-428b-8e08-7e1750fd13e7 None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:01:44.063327 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f765046-4e70-45cf-b82b-9e254199412e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:01:44.064143 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f765046-4e70-45cf-b82b-9e254199412e None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:44.064501 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f765046-4e70-45cf-b82b-9e254199412e None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:44.064780 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f765046-4e70-45cf-b82b-9e254199412e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:44.071749 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f765046-4e70-45cf-b82b-9e254199412e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:01:44.073235 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f765046-4e70-45cf-b82b-9e254199412e None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:01:44.087329 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application [None req-5f765046-4e70-45cf-b82b-9e254199412e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 12:01:44.087329 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:01:44.087329 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 12:01:44.087329 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 12:01:44.087329 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 12:01:44.087329 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 12:01:44.087329 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 12:01:44.087329 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 12:01:44.087329 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 12:01:44.087329 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 12:01:44.087329 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 12:01:44.087329 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 12:01:44.087329 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 12:01:44.087329 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 12:01:44.087329 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 12:01:44.088440 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 12:01:44.088440 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 12:01:44.088440 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 12:01:44.088440 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 12:01:44.088440 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 12:01:44.088440 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 12:01:44.088440 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.get_or_create( May 19 12:01:44.088440 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 12:01:44.088440 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application with Lock( May 19 12:01:44.088440 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 12:01:44.088440 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._enter() May 19 12:01:44.088440 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 12:01:44.088440 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 12:01:44.088440 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 12:01:44.088440 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.creator() May 19 12:01:44.088440 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 12:01:44.088440 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application created_value = creator( May 19 12:01:44.089756 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 12:01:44.089756 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 12:01:44.089756 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 12:01:44.089756 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 12:01:44.089756 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 12:01:44.089756 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 12:01:44.089756 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 12:01:44.089756 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application May 19 12:01:44.089756 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 623/1258] 10.222.0.23 () {56 vars in 1111 bytes} [Thu May 19 12:01:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 12:01:44.094404 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8e3ed3-baf9-4a63-88eb-20f1e5c58f73 None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:01:44.194471 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8e3ed3-baf9-4a63-88eb-20f1e5c58f73 None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:01:44.227880 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fa6bfd-c4a9-4f4f-993c-99afbd9eabd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:01:44.228537 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fa6bfd-c4a9-4f4f-993c-99afbd9eabd3 None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:44.228824 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fa6bfd-c4a9-4f4f-993c-99afbd9eabd3 None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:44.229101 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fa6bfd-c4a9-4f4f-993c-99afbd9eabd3 None admin] PATH_INFO: `/v3/projects` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:44.229876 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90fa6bfd-c4a9-4f4f-993c-99afbd9eabd3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:01:44.230190 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90fa6bfd-c4a9-4f4f-993c-99afbd9eabd3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:01:44.231638 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90fa6bfd-c4a9-4f4f-993c-99afbd9eabd3 None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:01:44.245268 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 636/1259] 10.222.0.23 () {56 vars in 1148 bytes} [Thu May 19 12:01:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:01:44.251814 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f765046-4e70-45cf-b82b-9e254199412e None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:01:44.286524 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:01:44.361655 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f765046-4e70-45cf-b82b-9e254199412e None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:01:44.394656 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de0c007-bfc1-48dd-90bc-e205cf01422d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:01:44.395298 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de0c007-bfc1-48dd-90bc-e205cf01422d None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:44.395574 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de0c007-bfc1-48dd-90bc-e205cf01422d None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:44.395870 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de0c007-bfc1-48dd-90bc-e205cf01422d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:44.396687 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de0c007-bfc1-48dd-90bc-e205cf01422d None admin] RBAC: Adding query filter params (role.id=9db5671c65d3414ea2682afa2ccf1403, scope.project.id=d70d25bf69314820b8aaecb65576978a, user.id=50f937eed4f3414fa6f19cab2a84eb5f) {{(pid=80539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:01:44.396999 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de0c007-bfc1-48dd-90bc-e205cf01422d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:01:44.398455 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de0c007-bfc1-48dd-90bc-e205cf01422d None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:01:44.404200 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 624/1260] 10.222.0.23 () {56 vars in 1374 bytes} [Thu May 19 12:01:44 2022] GET /identity/v3/role_assignments?user.id=50f937eed4f3414fa6f19cab2a84eb5f&scope.project.id=d70d25bf69314820b8aaecb65576978a&role.id=9db5671c65d3414ea2682afa2ccf1403 => generated 259 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:01:46.603310 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c0f85f-bf74-4c1e-b0ef-915e5eb850b2 None None] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:46.603892 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c0f85f-bf74-4c1e-b0ef-915e5eb850b2 None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:46.603892 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c0f85f-bf74-4c1e-b0ef-915e5eb850b2 None None] PATH_INFO: `/` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:46.605338 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 637/1261] 10.222.0.23 () {54 vars in 926 bytes} [Thu May 19 12:01:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 12:01:46.611756 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10f36b1-258f-46fb-a21c-1571fc72058b None None] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:46.612185 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10f36b1-258f-46fb-a21c-1571fc72058b None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:46.612982 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10f36b1-258f-46fb-a21c-1571fc72058b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:46.628706 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.auth.core [None req-b10f36b1-258f-46fb-a21c-1571fc72058b None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:01:46.629889 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-b10f36b1-258f-46fb-a21c-1571fc72058b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:01:46.695102 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 625/1262] 10.222.0.23 () {58 vars in 1023 bytes} [Thu May 19 12:01:46 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 12:01:46.729785 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c0f85f-bf74-4c1e-b0ef-915e5eb850b2 None None] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:01:46.741408 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:01:46.833841 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c0f85f-bf74-4c1e-b0ef-915e5eb850b2 None None] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:01:46.867845 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e009f93-eb43-4df9-974d-d9d25853f7b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:01:46.868634 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e009f93-eb43-4df9-974d-d9d25853f7b6 None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:46.868918 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e009f93-eb43-4df9-974d-d9d25853f7b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:46.869193 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e009f93-eb43-4df9-974d-d9d25853f7b6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:46.879742 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e009f93-eb43-4df9-974d-d9d25853f7b6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 12:01:46.887409 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application [None req-4e009f93-eb43-4df9-974d-d9d25853f7b6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 12:01:46.887409 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:01:46.887409 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 12:01:46.887409 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 12:01:46.887409 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 12:01:46.887409 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 12:01:46.887409 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 12:01:46.887409 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 12:01:46.887409 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 12:01:46.887409 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 12:01:46.887409 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 12:01:46.887409 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 12:01:46.887409 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 12:01:46.887409 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 12:01:46.887409 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 12:01:46.887409 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 12:01:46.889775 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 12:01:46.889775 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 12:01:46.889775 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 12:01:46.889775 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 12:01:46.889775 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 12:01:46.889775 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 12:01:46.889775 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 12:01:46.889775 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 12:01:46.889775 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 12:01:46.889775 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.get_or_create( May 19 12:01:46.889775 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 12:01:46.889775 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application with Lock( May 19 12:01:46.889775 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 12:01:46.889775 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._enter() May 19 12:01:46.889775 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 12:01:46.889775 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 12:01:46.890835 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 12:01:46.890835 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.creator() May 19 12:01:46.890835 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 12:01:46.890835 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application created_value = creator( May 19 12:01:46.890835 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 12:01:46.890835 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 12:01:46.890835 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 12:01:46.890835 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 12:01:46.890835 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 12:01:46.890835 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 12:01:46.890835 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 12:01:46.890835 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application May 19 12:01:46.890835 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 638/1263] 10.222.0.23 () {56 vars in 1096 bytes} [Thu May 19 12:01:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 12:01:46.895152 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10f36b1-258f-46fb-a21c-1571fc72058b None None] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:01:46.914292 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:01:46.997665 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10f36b1-258f-46fb-a21c-1571fc72058b None None] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:01:47.033764 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59229c2a-40ea-4e16-9b9d-ec20809e6bea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:01:47.034426 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59229c2a-40ea-4e16-9b9d-ec20809e6bea None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:47.034762 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59229c2a-40ea-4e16-9b9d-ec20809e6bea None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:47.035038 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59229c2a-40ea-4e16-9b9d-ec20809e6bea None admin] PATH_INFO: `/v3/roles` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:47.035843 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59229c2a-40ea-4e16-9b9d-ec20809e6bea None admin] RBAC: Adding query filter params (name=admin) {{(pid=80539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:01:47.036191 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59229c2a-40ea-4e16-9b9d-ec20809e6bea None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:01:47.037638 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59229c2a-40ea-4e16-9b9d-ec20809e6bea None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:01:47.046081 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 626/1264] 10.222.0.23 () {56 vars in 1099 bytes} [Thu May 19 12:01:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:01:47.052109 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e009f93-eb43-4df9-974d-d9d25853f7b6 None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:01:47.121141 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:01:47.154945 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e009f93-eb43-4df9-974d-d9d25853f7b6 None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:01:47.187588 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df09e51-4c23-4c8f-aeef-ae9663b249a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:01:47.188406 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df09e51-4c23-4c8f-aeef-ae9663b249a6 None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:47.188691 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df09e51-4c23-4c8f-aeef-ae9663b249a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:47.188968 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df09e51-4c23-4c8f-aeef-ae9663b249a6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:47.199759 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df09e51-4c23-4c8f-aeef-ae9663b249a6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:01:47.202152 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df09e51-4c23-4c8f-aeef-ae9663b249a6 None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:01:47.219226 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application [None req-5df09e51-4c23-4c8f-aeef-ae9663b249a6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 12:01:47.219226 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:01:47.219226 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 12:01:47.219226 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 12:01:47.219226 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 12:01:47.219226 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 12:01:47.219226 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 12:01:47.219226 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 12:01:47.219226 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 12:01:47.219226 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 12:01:47.219226 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 12:01:47.219226 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 12:01:47.219226 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 12:01:47.219226 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 12:01:47.219226 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 12:01:47.220345 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 12:01:47.220345 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 12:01:47.220345 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 12:01:47.220345 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 12:01:47.220345 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 12:01:47.220345 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 12:01:47.220345 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.get_or_create( May 19 12:01:47.220345 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 12:01:47.220345 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application with Lock( May 19 12:01:47.220345 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 12:01:47.220345 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._enter() May 19 12:01:47.220345 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 12:01:47.220345 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 12:01:47.220345 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 12:01:47.220345 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.creator() May 19 12:01:47.220345 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 12:01:47.220345 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application created_value = creator( May 19 12:01:47.221482 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 12:01:47.221482 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 12:01:47.221482 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 12:01:47.221482 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application May 19 12:01:47.221482 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 639/1265] 10.222.0.23 () {56 vars in 1108 bytes} [Thu May 19 12:01:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 12:01:47.226281 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59229c2a-40ea-4e16-9b9d-ec20809e6bea None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:01:47.307484 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:01:47.327347 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59229c2a-40ea-4e16-9b9d-ec20809e6bea None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:01:47.360175 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a085d4-5c36-49ce-84fd-2e78849d284f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:01:47.360869 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a085d4-5c36-49ce-84fd-2e78849d284f None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:47.361158 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a085d4-5c36-49ce-84fd-2e78849d284f None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:47.361440 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a085d4-5c36-49ce-84fd-2e78849d284f None admin] PATH_INFO: `/v3/domains` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:47.362169 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a085d4-5c36-49ce-84fd-2e78849d284f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:01:47.362480 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a085d4-5c36-49ce-84fd-2e78849d284f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:01:47.363876 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a085d4-5c36-49ce-84fd-2e78849d284f None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:01:47.376186 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 627/1266] 10.222.0.23 () {56 vars in 1109 bytes} [Thu May 19 12:01:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:01:47.381938 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df09e51-4c23-4c8f-aeef-ae9663b249a6 None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:01:47.481603 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df09e51-4c23-4c8f-aeef-ae9663b249a6 None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:01:47.501314 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:01:47.515264 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15400b6-4b34-49c1-939c-b5bd3dc6c227 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:01:47.516088 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15400b6-4b34-49c1-939c-b5bd3dc6c227 None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:47.516373 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15400b6-4b34-49c1-939c-b5bd3dc6c227 None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:47.516651 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15400b6-4b34-49c1-939c-b5bd3dc6c227 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:47.523971 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15400b6-4b34-49c1-939c-b5bd3dc6c227 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:01:47.525446 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15400b6-4b34-49c1-939c-b5bd3dc6c227 None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:01:47.539235 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application [None req-b15400b6-4b34-49c1-939c-b5bd3dc6c227 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 19 12:01:47.539235 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:01:47.539235 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 12:01:47.539235 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 12:01:47.539235 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 12:01:47.539235 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 12:01:47.539235 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 12:01:47.539235 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 12:01:47.539235 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 12:01:47.539235 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 12:01:47.539235 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 12:01:47.539235 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 12:01:47.539235 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 12:01:47.539235 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 12:01:47.539235 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 12:01:47.539235 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 12:01:47.540431 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 12:01:47.540431 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 12:01:47.540431 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 12:01:47.540431 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 12:01:47.540431 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 12:01:47.540431 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 12:01:47.540431 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 12:01:47.540431 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 12:01:47.540431 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 12:01:47.540431 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.get_or_create( May 19 12:01:47.540431 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 12:01:47.540431 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application with Lock( May 19 12:01:47.540431 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 12:01:47.540431 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._enter() May 19 12:01:47.540431 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 12:01:47.540431 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 12:01:47.540431 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 12:01:47.540431 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.creator() May 19 12:01:47.541699 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 12:01:47.541699 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application created_value = creator( May 19 12:01:47.541699 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 12:01:47.541699 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 12:01:47.541699 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 12:01:47.541699 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 12:01:47.541699 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 12:01:47.541699 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 12:01:47.541699 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 19 12:01:47.541699 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application May 19 12:01:47.541699 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 640/1267] 10.222.0.23 () {56 vars in 1108 bytes} [Thu May 19 12:01:47 2022] GET /identity/v3/users/placement => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 12:01:47.546961 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a085d4-5c36-49ce-84fd-2e78849d284f None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:01:47.646899 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a085d4-5c36-49ce-84fd-2e78849d284f None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:01:47.679545 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d873338-a0a4-4efe-b4c6-c075f1b4e733 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:01:47.680242 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d873338-a0a4-4efe-b4c6-c075f1b4e733 None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:47.680524 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d873338-a0a4-4efe-b4c6-c075f1b4e733 None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:47.680799 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d873338-a0a4-4efe-b4c6-c075f1b4e733 None admin] PATH_INFO: `/v3/users` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:47.681623 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d873338-a0a4-4efe-b4c6-c075f1b4e733 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:01:47.681927 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d873338-a0a4-4efe-b4c6-c075f1b4e733 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:01:47.683349 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d873338-a0a4-4efe-b4c6-c075f1b4e733 None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:01:47.692844 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:01:47.701410 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 628/1268] 10.222.0.23 () {56 vars in 1143 bytes} [Thu May 19 12:01:47 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:01:47.707592 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15400b6-4b34-49c1-939c-b5bd3dc6c227 None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:01:47.809729 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15400b6-4b34-49c1-939c-b5bd3dc6c227 None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:01:47.845575 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290ccd23-4611-4ad2-9b05-10679802d2ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:01:47.846369 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290ccd23-4611-4ad2-9b05-10679802d2ab None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:47.846657 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290ccd23-4611-4ad2-9b05-10679802d2ab None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:47.846961 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290ccd23-4611-4ad2-9b05-10679802d2ab None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:47.858748 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-290ccd23-4611-4ad2-9b05-10679802d2ab None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:01:47.867397 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-290ccd23-4611-4ad2-9b05-10679802d2ab None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:01:47.878693 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application [None req-290ccd23-4611-4ad2-9b05-10679802d2ab None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 12:01:47.878693 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:01:47.878693 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 12:01:47.878693 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 12:01:47.878693 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 12:01:47.878693 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 12:01:47.878693 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 12:01:47.878693 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 12:01:47.878693 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 12:01:47.878693 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 12:01:47.878693 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 12:01:47.878693 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 12:01:47.878693 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 12:01:47.878693 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 12:01:47.878693 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 12:01:47.879643 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 12:01:47.879643 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 12:01:47.879643 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 12:01:47.879643 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 12:01:47.879643 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 12:01:47.879643 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 12:01:47.879643 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.get_or_create( May 19 12:01:47.879643 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 12:01:47.879643 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application with Lock( May 19 12:01:47.879643 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 12:01:47.879643 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._enter() May 19 12:01:47.879643 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 12:01:47.879643 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 12:01:47.879643 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 12:01:47.879643 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.creator() May 19 12:01:47.879643 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 12:01:47.879643 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application created_value = creator( May 19 12:01:47.880585 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 12:01:47.880585 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 12:01:47.880585 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 12:01:47.880585 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application May 19 12:01:47.880585 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 641/1269] 10.222.0.23 () {56 vars in 1108 bytes} [Thu May 19 12:01:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 12:01:47.885730 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d873338-a0a4-4efe-b4c6-c075f1b4e733 None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:01:47.989658 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d873338-a0a4-4efe-b4c6-c075f1b4e733 None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:01:48.023649 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f464fdc3-51e0-44ec-b36a-d68ee90a3f59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:01:48.024341 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f464fdc3-51e0-44ec-b36a-d68ee90a3f59 None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:48.024626 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f464fdc3-51e0-44ec-b36a-d68ee90a3f59 None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:48.024902 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f464fdc3-51e0-44ec-b36a-d68ee90a3f59 None admin] PATH_INFO: `/v3/domains` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:48.025627 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f464fdc3-51e0-44ec-b36a-d68ee90a3f59 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:01:48.025935 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f464fdc3-51e0-44ec-b36a-d68ee90a3f59 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:01:48.027320 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f464fdc3-51e0-44ec-b36a-d68ee90a3f59 None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:01:48.039156 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 629/1270] 10.222.0.23 () {56 vars in 1109 bytes} [Thu May 19 12:01:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:01:48.045752 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290ccd23-4611-4ad2-9b05-10679802d2ab None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:01:48.080113 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:01:48.147059 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290ccd23-4611-4ad2-9b05-10679802d2ab None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:01:48.182329 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106c5d4f-976b-48e2-9bec-149f01cff72f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:01:48.183119 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106c5d4f-976b-48e2-9bec-149f01cff72f None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:48.183403 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106c5d4f-976b-48e2-9bec-149f01cff72f None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:48.183678 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106c5d4f-976b-48e2-9bec-149f01cff72f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:48.190506 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106c5d4f-976b-48e2-9bec-149f01cff72f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:01:48.191982 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106c5d4f-976b-48e2-9bec-149f01cff72f None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:01:48.206037 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application [None req-106c5d4f-976b-48e2-9bec-149f01cff72f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 12:01:48.206037 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:01:48.206037 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 12:01:48.206037 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 12:01:48.206037 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 12:01:48.206037 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 12:01:48.206037 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 12:01:48.206037 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 12:01:48.206037 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 12:01:48.206037 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 12:01:48.206037 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 12:01:48.206037 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 12:01:48.206037 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 12:01:48.206037 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 12:01:48.206037 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 12:01:48.207193 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 12:01:48.207193 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 12:01:48.207193 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 12:01:48.207193 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 12:01:48.207193 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 12:01:48.207193 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 12:01:48.207193 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.get_or_create( May 19 12:01:48.207193 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 12:01:48.207193 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application with Lock( May 19 12:01:48.207193 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 12:01:48.207193 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._enter() May 19 12:01:48.207193 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 12:01:48.207193 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 12:01:48.207193 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 12:01:48.207193 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.creator() May 19 12:01:48.207193 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 12:01:48.207193 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application created_value = creator( May 19 12:01:48.208140 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 12:01:48.208140 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 12:01:48.208140 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 12:01:48.208140 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 12:01:48.208140 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 12:01:48.208140 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 12:01:48.208140 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 12:01:48.208140 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application May 19 12:01:48.208140 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 642/1271] 10.222.0.23 () {56 vars in 1111 bytes} [Thu May 19 12:01:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 12:01:48.213084 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f464fdc3-51e0-44ec-b36a-d68ee90a3f59 None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:01:48.273067 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:01:48.315904 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f464fdc3-51e0-44ec-b36a-d68ee90a3f59 None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:01:48.349516 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9598ad4-c443-4f41-9e6d-8f11cdfb2cd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:01:48.350172 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9598ad4-c443-4f41-9e6d-8f11cdfb2cd4 None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:48.350456 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9598ad4-c443-4f41-9e6d-8f11cdfb2cd4 None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:48.350734 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9598ad4-c443-4f41-9e6d-8f11cdfb2cd4 None admin] PATH_INFO: `/v3/projects` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:48.351518 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9598ad4-c443-4f41-9e6d-8f11cdfb2cd4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:01:48.351855 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9598ad4-c443-4f41-9e6d-8f11cdfb2cd4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:01:48.353286 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9598ad4-c443-4f41-9e6d-8f11cdfb2cd4 None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:01:48.365731 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 630/1272] 10.222.0.23 () {56 vars in 1148 bytes} [Thu May 19 12:01:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:01:48.371824 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106c5d4f-976b-48e2-9bec-149f01cff72f None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:01:48.464137 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:01:48.473534 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106c5d4f-976b-48e2-9bec-149f01cff72f None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:01:48.506731 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e74eac-fc2e-44c3-8071-a2179d1e97d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:01:48.507608 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e74eac-fc2e-44c3-8071-a2179d1e97d4 None admin] REQUEST_METHOD: `PUT` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:48.507954 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e74eac-fc2e-44c3-8071-a2179d1e97d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:48.508213 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e74eac-fc2e-44c3-8071-a2179d1e97d4 None admin] PATH_INFO: `/v3/projects/d70d25bf69314820b8aaecb65576978a/users/50f937eed4f3414fa6f19cab2a84eb5f/roles/9db5671c65d3414ea2682afa2ccf1403` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:48.510134 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e74eac-fc2e-44c3-8071-a2179d1e97d4 None admin] RBAC: Authorizing `identity:create_grant(project_id=d70d25bf69314820b8aaecb65576978a, user_id=50f937eed4f3414fa6f19cab2a84eb5f, role_id=9db5671c65d3414ea2682afa2ccf1403)` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:01:48.511571 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e74eac-fc2e-44c3-8071-a2179d1e97d4 None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:01:48.599176 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 643/1273] 10.222.0.23 () {58 vars in 1439 bytes} [Thu May 19 12:01:48 2022] PUT /identity/v3/projects/d70d25bf69314820b8aaecb65576978a/users/50f937eed4f3414fa6f19cab2a84eb5f/roles/9db5671c65d3414ea2682afa2ccf1403 => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:01:50.838673 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfadf624-47e9-48a5-a1be-1c42647b7b87 None None] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:50.839224 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfadf624-47e9-48a5-a1be-1c42647b7b87 None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:50.839320 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfadf624-47e9-48a5-a1be-1c42647b7b87 None None] PATH_INFO: `/` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:50.840686 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 631/1274] 10.222.0.23 () {54 vars in 926 bytes} [Thu May 19 12:01:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 12:01:50.846916 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a77f84c-a2a9-41b1-aab3-828ea24d7e71 None None] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:50.847658 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a77f84c-a2a9-41b1-aab3-828ea24d7e71 None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:50.847658 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a77f84c-a2a9-41b1-aab3-828ea24d7e71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:50.865430 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.auth.core [None req-7a77f84c-a2a9-41b1-aab3-828ea24d7e71 None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:01:50.866667 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-7a77f84c-a2a9-41b1-aab3-828ea24d7e71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:01:50.938313 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 644/1275] 10.222.0.23 () {58 vars in 1023 bytes} [Thu May 19 12:01:50 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 12:01:50.962736 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfadf624-47e9-48a5-a1be-1c42647b7b87 None None] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:01:51.064410 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfadf624-47e9-48a5-a1be-1c42647b7b87 None None] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:01:51.083494 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:01:51.101637 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81511d54-3704-4d61-a535-4225827c3cbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:01:51.102365 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81511d54-3704-4d61-a535-4225827c3cbb None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:51.102647 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81511d54-3704-4d61-a535-4225827c3cbb None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:51.102928 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81511d54-3704-4d61-a535-4225827c3cbb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:51.113941 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81511d54-3704-4d61-a535-4225827c3cbb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 12:01:51.121568 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application [None req-81511d54-3704-4d61-a535-4225827c3cbb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 12:01:51.121568 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:01:51.121568 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 12:01:51.121568 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 12:01:51.121568 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 12:01:51.121568 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 12:01:51.121568 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 12:01:51.121568 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 12:01:51.121568 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 12:01:51.121568 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 12:01:51.121568 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 12:01:51.121568 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 12:01:51.121568 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 12:01:51.121568 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 12:01:51.121568 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 12:01:51.121568 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 12:01:51.124300 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 12:01:51.124300 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 12:01:51.124300 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 12:01:51.124300 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 12:01:51.124300 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 12:01:51.124300 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 12:01:51.124300 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 12:01:51.124300 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 12:01:51.124300 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 12:01:51.124300 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.get_or_create( May 19 12:01:51.124300 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 12:01:51.124300 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application with Lock( May 19 12:01:51.124300 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 12:01:51.124300 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._enter() May 19 12:01:51.124300 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 12:01:51.124300 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 12:01:51.125425 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 12:01:51.125425 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.creator() May 19 12:01:51.125425 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 12:01:51.125425 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application created_value = creator( May 19 12:01:51.125425 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 12:01:51.125425 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 12:01:51.125425 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 12:01:51.125425 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 12:01:51.125425 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 12:01:51.125425 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 12:01:51.125425 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 12:01:51.125425 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application May 19 12:01:51.125425 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 632/1276] 10.222.0.23 () {56 vars in 1096 bytes} [Thu May 19 12:01:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 12:01:51.129737 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a77f84c-a2a9-41b1-aab3-828ea24d7e71 None None] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:01:51.212050 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:01:51.230360 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a77f84c-a2a9-41b1-aab3-828ea24d7e71 None None] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:01:51.265719 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389c50c2-2c3d-4d48-95c4-7b7524b03e21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:01:51.266390 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389c50c2-2c3d-4d48-95c4-7b7524b03e21 None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:51.266681 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389c50c2-2c3d-4d48-95c4-7b7524b03e21 None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:51.266964 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389c50c2-2c3d-4d48-95c4-7b7524b03e21 None admin] PATH_INFO: `/v3/roles` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:51.267723 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389c50c2-2c3d-4d48-95c4-7b7524b03e21 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:01:51.268063 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389c50c2-2c3d-4d48-95c4-7b7524b03e21 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:01:51.269447 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389c50c2-2c3d-4d48-95c4-7b7524b03e21 None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:01:51.277347 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 645/1277] 10.222.0.23 () {56 vars in 1099 bytes} [Thu May 19 12:01:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:01:51.283932 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81511d54-3704-4d61-a535-4225827c3cbb None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:01:51.381774 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81511d54-3704-4d61-a535-4225827c3cbb None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:01:51.416519 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63bb1dce-4e31-433e-98da-50a16d29e23e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:01:51.417320 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bb1dce-4e31-433e-98da-50a16d29e23e None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:51.417603 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bb1dce-4e31-433e-98da-50a16d29e23e None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:51.417874 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bb1dce-4e31-433e-98da-50a16d29e23e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:51.428821 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63bb1dce-4e31-433e-98da-50a16d29e23e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:01:51.430266 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63bb1dce-4e31-433e-98da-50a16d29e23e None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:01:51.447103 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application [None req-63bb1dce-4e31-433e-98da-50a16d29e23e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 12:01:51.447103 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:01:51.447103 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 12:01:51.447103 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 12:01:51.447103 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 12:01:51.447103 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 12:01:51.447103 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 12:01:51.447103 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 12:01:51.447103 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 12:01:51.447103 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 12:01:51.447103 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 12:01:51.447103 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 12:01:51.447103 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 12:01:51.447103 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 12:01:51.447103 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 12:01:51.450293 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 12:01:51.450293 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 12:01:51.450293 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 12:01:51.450293 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 12:01:51.450293 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 12:01:51.450293 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 12:01:51.450293 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.get_or_create( May 19 12:01:51.450293 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 12:01:51.450293 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application with Lock( May 19 12:01:51.450293 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 12:01:51.450293 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._enter() May 19 12:01:51.450293 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 12:01:51.450293 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 12:01:51.450293 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 12:01:51.450293 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.creator() May 19 12:01:51.450293 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 12:01:51.450293 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application created_value = creator( May 19 12:01:51.452412 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 12:01:51.452412 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 12:01:51.452412 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 12:01:51.452412 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application May 19 12:01:51.452412 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 633/1278] 10.222.0.23 () {56 vars in 1108 bytes} [Thu May 19 12:01:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 12:01:51.454871 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389c50c2-2c3d-4d48-95c4-7b7524b03e21 None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:01:51.555685 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389c50c2-2c3d-4d48-95c4-7b7524b03e21 None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:01:51.589914 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2cc8ff1-0884-4b27-a283-54018992b26e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:01:51.590573 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cc8ff1-0884-4b27-a283-54018992b26e None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:51.590856 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cc8ff1-0884-4b27-a283-54018992b26e None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:51.591131 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cc8ff1-0884-4b27-a283-54018992b26e None admin] PATH_INFO: `/v3/domains` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:51.592503 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2cc8ff1-0884-4b27-a283-54018992b26e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:01:51.592807 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2cc8ff1-0884-4b27-a283-54018992b26e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:01:51.594185 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2cc8ff1-0884-4b27-a283-54018992b26e None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:01:51.605788 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 646/1279] 10.222.0.23 () {56 vars in 1109 bytes} [Thu May 19 12:01:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:01:51.611821 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63bb1dce-4e31-433e-98da-50a16d29e23e None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:01:51.643986 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:01:51.711732 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63bb1dce-4e31-433e-98da-50a16d29e23e None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:01:51.745362 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee40ded-f351-43b7-8db8-f9f41aa75d42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:01:51.746151 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee40ded-f351-43b7-8db8-f9f41aa75d42 None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:51.746433 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee40ded-f351-43b7-8db8-f9f41aa75d42 None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:51.746707 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee40ded-f351-43b7-8db8-f9f41aa75d42 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:51.755592 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee40ded-f351-43b7-8db8-f9f41aa75d42 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:01:51.757053 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee40ded-f351-43b7-8db8-f9f41aa75d42 None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:01:51.771707 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application [None req-eee40ded-f351-43b7-8db8-f9f41aa75d42 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 19 12:01:51.771707 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:01:51.771707 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 12:01:51.771707 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 12:01:51.771707 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 12:01:51.771707 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 12:01:51.771707 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 12:01:51.771707 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 12:01:51.771707 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 12:01:51.771707 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 12:01:51.771707 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 12:01:51.771707 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 12:01:51.771707 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 12:01:51.771707 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 12:01:51.771707 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 12:01:51.771707 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 12:01:51.772906 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 12:01:51.772906 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 12:01:51.772906 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 12:01:51.772906 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 12:01:51.772906 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 12:01:51.772906 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 12:01:51.772906 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 12:01:51.772906 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 12:01:51.772906 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 12:01:51.772906 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.get_or_create( May 19 12:01:51.772906 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 12:01:51.772906 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application with Lock( May 19 12:01:51.772906 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 12:01:51.772906 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._enter() May 19 12:01:51.772906 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 12:01:51.772906 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 12:01:51.772906 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 12:01:51.772906 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.creator() May 19 12:01:51.774079 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 12:01:51.774079 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application created_value = creator( May 19 12:01:51.774079 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 12:01:51.774079 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 12:01:51.774079 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 12:01:51.774079 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 12:01:51.774079 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 12:01:51.774079 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 12:01:51.774079 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 19 12:01:51.774079 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application May 19 12:01:51.774079 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 634/1280] 10.222.0.23 () {56 vars in 1108 bytes} [Thu May 19 12:01:51 2022] GET /identity/v3/users/placement => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 12:01:51.778798 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2cc8ff1-0884-4b27-a283-54018992b26e None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:01:51.782694 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:01:51.880413 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2cc8ff1-0884-4b27-a283-54018992b26e None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:01:51.914856 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8870e9-d775-48ab-8ef7-1c2112affb35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:01:51.915456 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8870e9-d775-48ab-8ef7-1c2112affb35 None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:51.915746 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8870e9-d775-48ab-8ef7-1c2112affb35 None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:51.916063 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8870e9-d775-48ab-8ef7-1c2112affb35 None admin] PATH_INFO: `/v3/users` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:51.916901 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8870e9-d775-48ab-8ef7-1c2112affb35 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:01:51.917216 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8870e9-d775-48ab-8ef7-1c2112affb35 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:01:51.918666 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8870e9-d775-48ab-8ef7-1c2112affb35 None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:01:51.937731 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 647/1281] 10.222.0.23 () {56 vars in 1143 bytes} [Thu May 19 12:01:51 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:01:51.943442 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee40ded-f351-43b7-8db8-f9f41aa75d42 None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:01:52.035997 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:01:52.045940 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee40ded-f351-43b7-8db8-f9f41aa75d42 None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:01:52.080331 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983ddcca-db2f-41ac-b7ef-c51bd1e0a843 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:01:52.080977 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983ddcca-db2f-41ac-b7ef-c51bd1e0a843 None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:52.081260 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983ddcca-db2f-41ac-b7ef-c51bd1e0a843 None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:52.081535 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983ddcca-db2f-41ac-b7ef-c51bd1e0a843 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:52.148240 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983ddcca-db2f-41ac-b7ef-c51bd1e0a843 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:01:52.149696 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983ddcca-db2f-41ac-b7ef-c51bd1e0a843 None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:01:52.237822 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:01:52.247994 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 635/1282] 10.222.0.23 () {58 vars in 1282 bytes} [Thu May 19 12:01:51 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 12:01:52.254667 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8870e9-d775-48ab-8ef7-1c2112affb35 None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:01:52.317604 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:01:52.358544 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8870e9-d775-48ab-8ef7-1c2112affb35 None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:01:52.392986 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4472edaa-5c10-454a-b1f8-d72171ab45de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:01:52.393783 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4472edaa-5c10-454a-b1f8-d72171ab45de None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:52.394064 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4472edaa-5c10-454a-b1f8-d72171ab45de None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:52.394340 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4472edaa-5c10-454a-b1f8-d72171ab45de None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:52.405314 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4472edaa-5c10-454a-b1f8-d72171ab45de None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:01:52.406786 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4472edaa-5c10-454a-b1f8-d72171ab45de None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:01:52.424418 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application [None req-4472edaa-5c10-454a-b1f8-d72171ab45de None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 12:01:52.424418 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:01:52.424418 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 12:01:52.424418 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 12:01:52.424418 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 12:01:52.424418 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 12:01:52.424418 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 12:01:52.424418 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 12:01:52.424418 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 12:01:52.424418 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 12:01:52.424418 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 12:01:52.424418 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 12:01:52.424418 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 12:01:52.424418 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 12:01:52.424418 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 12:01:52.425468 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 12:01:52.425468 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 12:01:52.425468 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 12:01:52.425468 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 12:01:52.425468 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 12:01:52.425468 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 12:01:52.425468 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.get_or_create( May 19 12:01:52.425468 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 12:01:52.425468 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application with Lock( May 19 12:01:52.425468 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 12:01:52.425468 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._enter() May 19 12:01:52.425468 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 12:01:52.425468 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 12:01:52.425468 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 12:01:52.425468 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.creator() May 19 12:01:52.425468 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 12:01:52.425468 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application created_value = creator( May 19 12:01:52.426573 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 12:01:52.426573 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 12:01:52.426573 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 12:01:52.426573 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application May 19 12:01:52.426573 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 648/1283] 10.222.0.23 () {56 vars in 1108 bytes} [Thu May 19 12:01:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 12:01:52.431484 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983ddcca-db2f-41ac-b7ef-c51bd1e0a843 None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:01:52.532672 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983ddcca-db2f-41ac-b7ef-c51bd1e0a843 None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:01:52.567202 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3b8e05-457e-4464-b54d-7f9b6a9ff734 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:01:52.568067 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3b8e05-457e-4464-b54d-7f9b6a9ff734 None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:52.568355 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3b8e05-457e-4464-b54d-7f9b6a9ff734 None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:52.568632 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3b8e05-457e-4464-b54d-7f9b6a9ff734 None admin] PATH_INFO: `/v3/domains` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:52.569350 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3b8e05-457e-4464-b54d-7f9b6a9ff734 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:01:52.569659 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3b8e05-457e-4464-b54d-7f9b6a9ff734 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:01:52.571049 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3b8e05-457e-4464-b54d-7f9b6a9ff734 None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:01:52.583112 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 636/1284] 10.222.0.23 () {56 vars in 1109 bytes} [Thu May 19 12:01:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:01:52.588938 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4472edaa-5c10-454a-b1f8-d72171ab45de None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:01:52.692914 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4472edaa-5c10-454a-b1f8-d72171ab45de None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:01:52.707180 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:01:52.729303 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b41f6b-b4e8-4297-bc63-4141ea526867 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:01:52.730092 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b41f6b-b4e8-4297-bc63-4141ea526867 None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:52.730377 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b41f6b-b4e8-4297-bc63-4141ea526867 None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:52.730657 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b41f6b-b4e8-4297-bc63-4141ea526867 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:52.738522 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b41f6b-b4e8-4297-bc63-4141ea526867 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:01:52.740030 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b41f6b-b4e8-4297-bc63-4141ea526867 None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:01:52.754424 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application [None req-18b41f6b-b4e8-4297-bc63-4141ea526867 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 12:01:52.754424 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:01:52.754424 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 12:01:52.754424 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 12:01:52.754424 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 12:01:52.754424 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 12:01:52.754424 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 12:01:52.754424 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 12:01:52.754424 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 12:01:52.754424 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 12:01:52.754424 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 12:01:52.754424 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 12:01:52.754424 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 12:01:52.754424 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 12:01:52.754424 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 12:01:52.756197 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 12:01:52.756197 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 12:01:52.756197 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 12:01:52.756197 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 12:01:52.756197 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 12:01:52.756197 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 12:01:52.756197 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.get_or_create( May 19 12:01:52.756197 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 12:01:52.756197 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application with Lock( May 19 12:01:52.756197 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 12:01:52.756197 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._enter() May 19 12:01:52.756197 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 12:01:52.756197 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 12:01:52.756197 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 12:01:52.756197 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.creator() May 19 12:01:52.756197 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 12:01:52.756197 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application created_value = creator( May 19 12:01:52.757863 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 12:01:52.757863 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 12:01:52.757863 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 12:01:52.757863 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 12:01:52.757863 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 12:01:52.757863 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 12:01:52.757863 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 12:01:52.757863 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: ERROR keystone.server.flask.application May 19 12:01:52.757863 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 649/1285] 10.222.0.23 () {56 vars in 1111 bytes} [Thu May 19 12:01:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 12:01:52.761669 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3b8e05-457e-4464-b54d-7f9b6a9ff734 None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:01:52.814207 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:01:52.865161 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3b8e05-457e-4464-b54d-7f9b6a9ff734 None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:01:52.899573 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc184e4-263a-423c-9535-5e438d829cdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:01:52.900277 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc184e4-263a-423c-9535-5e438d829cdc None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:52.900562 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc184e4-263a-423c-9535-5e438d829cdc None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:52.900839 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc184e4-263a-423c-9535-5e438d829cdc None admin] PATH_INFO: `/v3/projects` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:52.901673 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc184e4-263a-423c-9535-5e438d829cdc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:01:52.901986 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc184e4-263a-423c-9535-5e438d829cdc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:01:52.903421 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc184e4-263a-423c-9535-5e438d829cdc None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:01:52.919852 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 637/1286] 10.222.0.23 () {56 vars in 1148 bytes} [Thu May 19 12:01:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:01:52.923214 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b41f6b-b4e8-4297-bc63-4141ea526867 None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:01:53.028523 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b41f6b-b4e8-4297-bc63-4141ea526867 None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:01:53.064010 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad94a48-f611-4a40-9a6b-23017a5cdfe2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:01:53.064749 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad94a48-f611-4a40-9a6b-23017a5cdfe2 None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:53.064987 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad94a48-f611-4a40-9a6b-23017a5cdfe2 None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:53.065281 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad94a48-f611-4a40-9a6b-23017a5cdfe2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:53.066107 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad94a48-f611-4a40-9a6b-23017a5cdfe2 None admin] RBAC: Adding query filter params (role.id=9db5671c65d3414ea2682afa2ccf1403, scope.project.id=d70d25bf69314820b8aaecb65576978a, user.id=50f937eed4f3414fa6f19cab2a84eb5f) {{(pid=80540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:01:53.066425 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad94a48-f611-4a40-9a6b-23017a5cdfe2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:01:53.067898 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad94a48-f611-4a40-9a6b-23017a5cdfe2 None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:01:53.074770 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 650/1287] 10.222.0.23 () {56 vars in 1374 bytes} [Thu May 19 12:01:52 2022] GET /identity/v3/role_assignments?user.id=50f937eed4f3414fa6f19cab2a84eb5f&scope.project.id=d70d25bf69314820b8aaecb65576978a&role.id=9db5671c65d3414ea2682afa2ccf1403 => generated 608 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:01:55.387566 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1de57d-ac60-4475-a9e4-3a0e29917dc5 None None] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:55.388789 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1de57d-ac60-4475-a9e4-3a0e29917dc5 None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:55.388789 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1de57d-ac60-4475-a9e4-3a0e29917dc5 None None] PATH_INFO: `/` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:55.390361 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 638/1288] 10.222.0.23 () {54 vars in 926 bytes} [Thu May 19 12:01:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 12:01:55.395607 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a9d9ec-0563-4709-aab9-d607d699aaf2 None None] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:55.396069 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a9d9ec-0563-4709-aab9-d607d699aaf2 None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:55.396207 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a9d9ec-0563-4709-aab9-d607d699aaf2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:55.413095 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.auth.core [None req-d1a9d9ec-0563-4709-aab9-d607d699aaf2 None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:01:55.414347 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-d1a9d9ec-0563-4709-aab9-d607d699aaf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:01:55.444155 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:01:55.481147 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 651/1289] 10.222.0.23 () {58 vars in 1023 bytes} [Thu May 19 12:01:55 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 12:01:55.504180 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1de57d-ac60-4475-a9e4-3a0e29917dc5 None None] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:01:55.604829 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1de57d-ac60-4475-a9e4-3a0e29917dc5 None None] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:01:55.616503 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:01:55.640783 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7baf4bb1-582f-413d-b818-18dc064e8235 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:01:55.641504 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7baf4bb1-582f-413d-b818-18dc064e8235 None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:55.641783 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7baf4bb1-582f-413d-b818-18dc064e8235 None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:55.642055 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7baf4bb1-582f-413d-b818-18dc064e8235 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:55.643021 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7baf4bb1-582f-413d-b818-18dc064e8235 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:01:55.644416 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7baf4bb1-582f-413d-b818-18dc064e8235 None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:01:55.656152 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application [None req-7baf4bb1-582f-413d-b818-18dc064e8235 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. May 19 12:01:55.656152 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:01:55.656152 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 12:01:55.656152 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 12:01:55.656152 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 12:01:55.656152 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 12:01:55.656152 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 12:01:55.656152 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 12:01:55.656152 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 19 12:01:55.656152 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application May 19 12:01:55.656152 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 12:01:55.656152 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application May 19 12:01:55.656152 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:01:55.656152 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 12:01:55.656152 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 12:01:55.656152 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 12:01:55.656152 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 12:01:55.657286 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 12:01:55.657286 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 12:01:55.657286 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 12:01:55.657286 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 12:01:55.657286 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 12:01:55.657286 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 12:01:55.657286 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 12:01:55.657286 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 12:01:55.657286 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 12:01:55.657286 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 12:01:55.657286 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 12:01:55.657286 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 12:01:55.657286 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 12:01:55.657286 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 12:01:55.657286 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 12:01:55.657286 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.get_or_create( May 19 12:01:55.658137 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 12:01:55.658137 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application with Lock( May 19 12:01:55.658137 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 12:01:55.658137 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._enter() May 19 12:01:55.658137 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 12:01:55.658137 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 12:01:55.658137 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 12:01:55.658137 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.creator() May 19 12:01:55.658137 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 12:01:55.658137 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application created_value = creator( May 19 12:01:55.658137 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 12:01:55.658137 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 12:01:55.658137 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 19 12:01:55.658137 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application May 19 12:01:55.658137 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 639/1290] 10.222.0.23 () {56 vars in 1117 bytes} [Thu May 19 12:01:55 2022] GET /identity/v3/services/placement => generated 90 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 12:01:55.663790 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a9d9ec-0563-4709-aab9-d607d699aaf2 None None] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:01:55.766565 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a9d9ec-0563-4709-aab9-d607d699aaf2 None None] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:01:55.802475 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7378ab4c-59e0-44a4-a925-5d19de8c2aab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:01:55.803139 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7378ab4c-59e0-44a4-a925-5d19de8c2aab None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:55.803429 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7378ab4c-59e0-44a4-a925-5d19de8c2aab None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:55.803704 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7378ab4c-59e0-44a4-a925-5d19de8c2aab None admin] PATH_INFO: `/v3/services` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:55.804443 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7378ab4c-59e0-44a4-a925-5d19de8c2aab None admin] RBAC: Adding query filter params (name=placement) {{(pid=80540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:01:55.804750 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7378ab4c-59e0-44a4-a925-5d19de8c2aab None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:01:55.806130 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7378ab4c-59e0-44a4-a925-5d19de8c2aab None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:01:55.812164 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 652/1291] 10.222.0.23 () {56 vars in 1116 bytes} [Thu May 19 12:01:55 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:01:55.817933 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7baf4bb1-582f-413d-b818-18dc064e8235 None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:01:55.917713 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7baf4bb1-582f-413d-b818-18dc064e8235 None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:01:55.951616 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a9122a-29fa-4e80-b53a-f45897e94ad2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:01:55.952556 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a9122a-29fa-4e80-b53a-f45897e94ad2 None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:55.952835 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a9122a-29fa-4e80-b53a-f45897e94ad2 None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:55.953105 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a9122a-29fa-4e80-b53a-f45897e94ad2 None admin] PATH_INFO: `/v3/services` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:55.953804 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a9122a-29fa-4e80-b53a-f45897e94ad2 None admin] RBAC: Adding query filter params (type=placement) {{(pid=80539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:01:55.954111 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a9122a-29fa-4e80-b53a-f45897e94ad2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:01:55.955476 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a9122a-29fa-4e80-b53a-f45897e94ad2 None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:01:55.959987 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 640/1292] 10.222.0.23 () {56 vars in 1116 bytes} [Thu May 19 12:01:55 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:01:58.178329 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22792219-e5c3-49b4-9854-8f3bc69c4313 None None] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:58.178883 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22792219-e5c3-49b4-9854-8f3bc69c4313 None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:58.178883 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22792219-e5c3-49b4-9854-8f3bc69c4313 None None] PATH_INFO: `/` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:58.180434 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 653/1293] 10.222.0.23 () {54 vars in 926 bytes} [Thu May 19 12:01:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 12:01:58.185692 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c535c1-fdbf-4dc5-a9ed-132ac099efc1 None None] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:58.186215 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c535c1-fdbf-4dc5-a9ed-132ac099efc1 None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:58.186318 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c535c1-fdbf-4dc5-a9ed-132ac099efc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:58.203929 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.auth.core [None req-55c535c1-fdbf-4dc5-a9ed-132ac099efc1 None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:01:58.205076 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-55c535c1-fdbf-4dc5-a9ed-132ac099efc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:01:58.233574 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:01:58.269004 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 641/1294] 10.222.0.23 () {58 vars in 1023 bytes} [Thu May 19 12:01:58 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 12:01:58.291467 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22792219-e5c3-49b4-9854-8f3bc69c4313 None None] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:01:58.323970 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:01:58.395168 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22792219-e5c3-49b4-9854-8f3bc69c4313 None None] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:01:58.433251 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41807a95-d3a0-4c4e-a16e-66f39f3d7ee5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:01:58.433947 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41807a95-d3a0-4c4e-a16e-66f39f3d7ee5 None admin] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:58.434233 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41807a95-d3a0-4c4e-a16e-66f39f3d7ee5 None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:58.434510 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41807a95-d3a0-4c4e-a16e-66f39f3d7ee5 None admin] PATH_INFO: `/v3/services` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:58.435244 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41807a95-d3a0-4c4e-a16e-66f39f3d7ee5 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:01:58.436784 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41807a95-d3a0-4c4e-a16e-66f39f3d7ee5 None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:01:58.553095 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 654/1295] 10.222.0.23 () {60 vars in 1141 bytes} [Thu May 19 12:01:58 2022] POST /identity/v3/services => generated 246 bytes in 262 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 12:02:00.869425 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564b0dea-58e6-46a0-8193-dce6e77e9267 None None] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:00.870028 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564b0dea-58e6-46a0-8193-dce6e77e9267 None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:00.870028 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564b0dea-58e6-46a0-8193-dce6e77e9267 None None] PATH_INFO: `/` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:00.871329 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 642/1296] 10.222.0.23 () {54 vars in 926 bytes} [Thu May 19 12:02:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 12:02:00.877687 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da98d3fe-e6c8-4604-9343-f29ecb8d4051 None None] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:00.878188 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da98d3fe-e6c8-4604-9343-f29ecb8d4051 None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:00.878279 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da98d3fe-e6c8-4604-9343-f29ecb8d4051 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:00.896820 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.auth.core [None req-da98d3fe-e6c8-4604-9343-f29ecb8d4051 None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:02:00.898133 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-da98d3fe-e6c8-4604-9343-f29ecb8d4051 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:02:00.983393 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 655/1297] 10.222.0.23 () {58 vars in 1023 bytes} [Thu May 19 12:02:00 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 12:02:01.007357 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564b0dea-58e6-46a0-8193-dce6e77e9267 None None] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:01.116143 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564b0dea-58e6-46a0-8193-dce6e77e9267 None None] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:01.151507 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da077997-7b54-4102-b5b5-a28f945cf354 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:01.152264 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da077997-7b54-4102-b5b5-a28f945cf354 None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:01.152554 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da077997-7b54-4102-b5b5-a28f945cf354 None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:01.152832 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da077997-7b54-4102-b5b5-a28f945cf354 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:01.153501 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da077997-7b54-4102-b5b5-a28f945cf354 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:02:01.155127 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da077997-7b54-4102-b5b5-a28f945cf354 None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:02:01.167292 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application [None req-da077997-7b54-4102-b5b5-a28f945cf354 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. May 19 12:02:01.167292 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:02:01.167292 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 12:02:01.167292 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 12:02:01.167292 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 12:02:01.167292 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 12:02:01.167292 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 12:02:01.167292 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 12:02:01.167292 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 19 12:02:01.167292 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application May 19 12:02:01.167292 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 12:02:01.167292 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application May 19 12:02:01.167292 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:02:01.167292 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 12:02:01.167292 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 12:02:01.167292 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 12:02:01.167292 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 12:02:01.168553 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 12:02:01.168553 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 12:02:01.168553 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 12:02:01.168553 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 12:02:01.168553 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 12:02:01.168553 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 12:02:01.168553 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 12:02:01.168553 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 12:02:01.168553 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 12:02:01.168553 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 12:02:01.168553 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 12:02:01.168553 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 12:02:01.168553 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 12:02:01.168553 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 12:02:01.168553 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 12:02:01.168553 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.get_or_create( May 19 12:02:01.170154 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 12:02:01.170154 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application with Lock( May 19 12:02:01.170154 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 12:02:01.170154 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._enter() May 19 12:02:01.170154 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 12:02:01.170154 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 12:02:01.170154 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 12:02:01.170154 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.creator() May 19 12:02:01.170154 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 12:02:01.170154 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application created_value = creator( May 19 12:02:01.170154 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 12:02:01.170154 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 12:02:01.170154 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 19 12:02:01.170154 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application May 19 12:02:01.170154 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 643/1298] 10.222.0.23 () {56 vars in 1117 bytes} [Thu May 19 12:02:01 2022] GET /identity/v3/services/placement => generated 90 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 12:02:01.175320 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da98d3fe-e6c8-4604-9343-f29ecb8d4051 None None] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:01.196559 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:02:01.283274 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da98d3fe-e6c8-4604-9343-f29ecb8d4051 None None] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:01.320240 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22483889-bc1f-482a-b6fa-e4c4395a54d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:01.320933 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22483889-bc1f-482a-b6fa-e4c4395a54d5 None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:01.321223 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22483889-bc1f-482a-b6fa-e4c4395a54d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:01.321505 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22483889-bc1f-482a-b6fa-e4c4395a54d5 None admin] PATH_INFO: `/v3/services` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:01.322221 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22483889-bc1f-482a-b6fa-e4c4395a54d5 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:02:01.322533 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22483889-bc1f-482a-b6fa-e4c4395a54d5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:02:01.323963 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22483889-bc1f-482a-b6fa-e4c4395a54d5 None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:02:01.329722 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 656/1299] 10.222.0.23 () {56 vars in 1116 bytes} [Thu May 19 12:02:01 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:02:01.336434 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da077997-7b54-4102-b5b5-a28f945cf354 None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:01.373424 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:02:01.444514 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da077997-7b54-4102-b5b5-a28f945cf354 None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:01.478887 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52c9ae6-95c5-4edb-be22-649a83e31029 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:01.479548 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52c9ae6-95c5-4edb-be22-649a83e31029 None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:01.479858 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52c9ae6-95c5-4edb-be22-649a83e31029 None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:01.480136 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52c9ae6-95c5-4edb-be22-649a83e31029 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:01.480934 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52c9ae6-95c5-4edb-be22-649a83e31029 None admin] RBAC: Adding query filter params (interface=public, service_id=824c727057684fe990545b09e7530899, region_id=RegionOne) {{(pid=80539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:02:01.481248 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52c9ae6-95c5-4edb-be22-649a83e31029 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:02:01.483418 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52c9ae6-95c5-4edb-be22-649a83e31029 None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:02:01.488781 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 644/1300] 10.222.0.23 () {56 vars in 1251 bytes} [Thu May 19 12:02:01 2022] GET /identity/v3/endpoints?service_id=824c727057684fe990545b09e7530899&interface=public®ion_id=RegionOne => generated 194 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:02:01.494656 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22483889-bc1f-482a-b6fa-e4c4395a54d5 None admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:01.579934 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:02:01.600085 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22483889-bc1f-482a-b6fa-e4c4395a54d5 None admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:01.636130 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436e2fd6-2782-4de8-aa99-6cf0761cf147 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:01.636801 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436e2fd6-2782-4de8-aa99-6cf0761cf147 None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:01.637087 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436e2fd6-2782-4de8-aa99-6cf0761cf147 None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:01.637366 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436e2fd6-2782-4de8-aa99-6cf0761cf147 None admin] PATH_INFO: `/v3/services` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:01.638047 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436e2fd6-2782-4de8-aa99-6cf0761cf147 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:02:01.639500 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436e2fd6-2782-4de8-aa99-6cf0761cf147 None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:02:01.647259 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 657/1301] 10.222.0.23 () {56 vars in 1087 bytes} [Thu May 19 12:02:01 2022] GET /identity/v3/services => generated 2191 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 12:02:03.957424 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0d55d7-f689-46fa-8218-7d43452e0264 None None] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:03.957950 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0d55d7-f689-46fa-8218-7d43452e0264 None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:03.958037 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0d55d7-f689-46fa-8218-7d43452e0264 None None] PATH_INFO: `/` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:03.959345 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 645/1302] 10.222.0.23 () {54 vars in 926 bytes} [Thu May 19 12:02:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 12:02:03.965012 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8f0e03-c7b9-418b-ad09-64e38c7513e1 None None] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:03.965499 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8f0e03-c7b9-418b-ad09-64e38c7513e1 None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:03.968661 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8f0e03-c7b9-418b-ad09-64e38c7513e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:03.985322 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.auth.core [None req-da8f0e03-c7b9-418b-ad09-64e38c7513e1 None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:02:03.986522 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-da8f0e03-c7b9-418b-ad09-64e38c7513e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:02:04.056940 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 658/1303] 10.222.0.23 () {58 vars in 1023 bytes} [Thu May 19 12:02:03 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 12:02:04.080410 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0d55d7-f689-46fa-8218-7d43452e0264 None None] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:04.180375 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:02:04.187304 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0d55d7-f689-46fa-8218-7d43452e0264 None None] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:04.224345 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7f2dc9-51f1-4794-bb34-0ee5a6367c0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:04.225121 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7f2dc9-51f1-4794-bb34-0ee5a6367c0c None admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:04.225405 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7f2dc9-51f1-4794-bb34-0ee5a6367c0c None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:04.225681 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7f2dc9-51f1-4794-bb34-0ee5a6367c0c None admin] PATH_INFO: `/v3/services/placement` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:04.226360 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7f2dc9-51f1-4794-bb34-0ee5a6367c0c None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:02:04.227728 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7f2dc9-51f1-4794-bb34-0ee5a6367c0c None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:02:04.243933 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application [None req-3a7f2dc9-51f1-4794-bb34-0ee5a6367c0c None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. May 19 12:02:04.243933 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:02:04.243933 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 12:02:04.243933 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 12:02:04.243933 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 12:02:04.243933 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 12:02:04.243933 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 12:02:04.243933 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 12:02:04.243933 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 19 12:02:04.243933 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application May 19 12:02:04.243933 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 12:02:04.243933 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application May 19 12:02:04.243933 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:02:04.243933 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 12:02:04.243933 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 12:02:04.243933 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 12:02:04.243933 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 12:02:04.245151 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 12:02:04.245151 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 12:02:04.245151 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 12:02:04.245151 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 12:02:04.245151 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 12:02:04.245151 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 12:02:04.245151 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 12:02:04.245151 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 12:02:04.245151 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 12:02:04.245151 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 12:02:04.245151 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 12:02:04.245151 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 12:02:04.245151 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 12:02:04.245151 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 12:02:04.245151 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 12:02:04.245151 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.get_or_create( May 19 12:02:04.246198 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 12:02:04.246198 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application with Lock( May 19 12:02:04.246198 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 12:02:04.246198 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self._enter() May 19 12:02:04.246198 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 12:02:04.246198 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 12:02:04.246198 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 12:02:04.246198 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application return self.creator() May 19 12:02:04.246198 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 12:02:04.246198 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application created_value = creator( May 19 12:02:04.246198 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 12:02:04.246198 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 12:02:04.246198 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 19 12:02:04.246198 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: ERROR keystone.server.flask.application May 19 12:02:04.246198 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 646/1304] 10.222.0.23 () {56 vars in 1117 bytes} [Thu May 19 12:02:04 2022] GET /identity/v3/services/placement => generated 90 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 12:02:04.251575 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8f0e03-c7b9-418b-ad09-64e38c7513e1 None None] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:04.332209 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:02:04.355927 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8f0e03-c7b9-418b-ad09-64e38c7513e1 None None] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:04.392547 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d11086-22cb-4b6e-a59c-e1392ee5437c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:04.393246 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d11086-22cb-4b6e-a59c-e1392ee5437c None admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:04.393514 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d11086-22cb-4b6e-a59c-e1392ee5437c None admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:04.393795 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d11086-22cb-4b6e-a59c-e1392ee5437c None admin] PATH_INFO: `/v3/services` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:04.394517 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d11086-22cb-4b6e-a59c-e1392ee5437c None admin] RBAC: Adding query filter params (name=placement) {{(pid=80540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:02:04.394828 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d11086-22cb-4b6e-a59c-e1392ee5437c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:02:04.396296 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d11086-22cb-4b6e-a59c-e1392ee5437c None admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:02:04.402095 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 659/1305] 10.222.0.23 () {56 vars in 1116 bytes} [Thu May 19 12:02:04 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:02:04.408165 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7f2dc9-51f1-4794-bb34-0ee5a6367c0c None admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:04.512280 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7f2dc9-51f1-4794-bb34-0ee5a6367c0c None admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:04.546287 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca620052-2c2f-48d5-8aab-cb688f22331f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:04.546935 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca620052-2c2f-48d5-8aab-cb688f22331f None admin] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:04.547221 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca620052-2c2f-48d5-8aab-cb688f22331f None admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:04.547500 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca620052-2c2f-48d5-8aab-cb688f22331f None admin] PATH_INFO: `/v3/endpoints` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:04.548280 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca620052-2c2f-48d5-8aab-cb688f22331f None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:02:04.549656 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca620052-2c2f-48d5-8aab-cb688f22331f None admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:02:04.638087 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 647/1306] 10.222.0.23 () {60 vars in 1144 bytes} [Thu May 19 12:02:04 2022] POST /identity/v3/endpoints => generated 331 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 12:02:14.398711 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=30 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:02:14.633124 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:08:15.504861 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7adaab0-55b8-4987-93c6-4722e7786123 None None] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:08:15.505421 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7adaab0-55b8-4987-93c6-4722e7786123 None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:08:15.505421 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7adaab0-55b8-4987-93c6-4722e7786123 None None] PATH_INFO: `/` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:08:15.512465 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96878ca0-2a00-413a-b8b3-216eec3e825e None None] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:08:15.512986 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96878ca0-2a00-413a-b8b3-216eec3e825e None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:08:15.513067 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96878ca0-2a00-413a-b8b3-216eec3e825e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:08:15.515206 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 660/1307] 10.222.0.23 () {54 vars in 926 bytes} [Thu May 19 12:08:15 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 12:08:15.702698 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.auth.core [None req-96878ca0-2a00-413a-b8b3-216eec3e825e None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:08:15.704824 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-96878ca0-2a00-413a-b8b3-216eec3e825e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:08:15.832944 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 648/1308] 10.222.0.23 () {58 vars in 1023 bytes} [Thu May 19 12:08:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 321 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 12:08:15.885661 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840d385c-8f9c-44c6-85a6-851303f78a99 None None] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:08:15.886223 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840d385c-8f9c-44c6-85a6-851303f78a99 None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:08:15.886676 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840d385c-8f9c-44c6-85a6-851303f78a99 None None] PATH_INFO: `/` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:08:15.888359 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 661/1309] 10.222.0.23 () {54 vars in 964 bytes} [Thu May 19 12:08:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 12:08:15.911427 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce9e1cb-910d-476e-9582-ab073c64eae1 None None] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:08:15.911899 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce9e1cb-910d-476e-9582-ab073c64eae1 None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:08:15.912336 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce9e1cb-910d-476e-9582-ab073c64eae1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:08:16.014733 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.auth.core [None req-5ce9e1cb-910d-476e-9582-ab073c64eae1 None None] MFA Rules not processed for user `1a00b3fa7497456895e5e0bf105c4fc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:08:16.017218 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-5ce9e1cb-910d-476e-9582-ab073c64eae1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:08:16.047309 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:08:16.126495 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 649/1310] 10.222.0.23 () {58 vars in 1061 bytes} [Thu May 19 12:08:15 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:08:16.135484 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840d385c-8f9c-44c6-85a6-851303f78a99 None None] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:08:16.137492 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-840d385c-8f9c-44c6-85a6-851303f78a99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:08:16.263591 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840d385c-8f9c-44c6-85a6-851303f78a99 None None] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:08:16.355007 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733d0db8-9230-49a8-be34-408a43cd8fc4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a00b3fa7497456895e5e0bf105c4fc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd70d25bf69314820b8aaecb65576978a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:08:16.355877 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733d0db8-9230-49a8-be34-408a43cd8fc4 service neutron] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:08:16.356276 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733d0db8-9230-49a8-be34-408a43cd8fc4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:08:16.356660 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733d0db8-9230-49a8-be34-408a43cd8fc4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:08:16.358544 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-733d0db8-9230-49a8-be34-408a43cd8fc4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:08:16.426080 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:08:16.493375 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733d0db8-9230-49a8-be34-408a43cd8fc4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:08:16.495041 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733d0db8-9230-49a8-be34-408a43cd8fc4 service neutron] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:08:16.520149 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 662/1311] 10.222.0.23 () {60 vars in 1365 bytes} [Thu May 19 12:08:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:08:19.145214 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7531fa-3b0b-4df6-b47c-2593e0f2f1e5 None None] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:08:19.145811 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7531fa-3b0b-4df6-b47c-2593e0f2f1e5 None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:08:19.145811 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7531fa-3b0b-4df6-b47c-2593e0f2f1e5 None None] PATH_INFO: `/` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:08:19.147309 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 650/1312] 10.222.0.23 () {54 vars in 926 bytes} [Thu May 19 12:08:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 12:08:19.153976 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a234d4e-3aa4-4b3d-bfdd-786ff41c8245 None None] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:08:19.155843 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a234d4e-3aa4-4b3d-bfdd-786ff41c8245 None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:08:19.163227 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a234d4e-3aa4-4b3d-bfdd-786ff41c8245 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:08:19.277762 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.auth.core [None req-6a234d4e-3aa4-4b3d-bfdd-786ff41c8245 None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:08:19.279298 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-6a234d4e-3aa4-4b3d-bfdd-786ff41c8245 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:08:19.403821 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 663/1313] 10.222.0.23 () {58 vars in 1023 bytes} [Thu May 19 12:08:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 248 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:08:19.435856 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7531fa-3b0b-4df6-b47c-2593e0f2f1e5 None None] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:08:19.437575 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-7a7531fa-3b0b-4df6-b47c-2593e0f2f1e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:08:19.572464 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7531fa-3b0b-4df6-b47c-2593e0f2f1e5 None None] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:08:19.672828 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:08:19.675674 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46eaa689-3300-41bd-9c3d-cb995f2a5136 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a00b3fa7497456895e5e0bf105c4fc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd70d25bf69314820b8aaecb65576978a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:08:19.676368 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46eaa689-3300-41bd-9c3d-cb995f2a5136 service neutron] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:08:19.676660 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46eaa689-3300-41bd-9c3d-cb995f2a5136 service neutron] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:08:19.676939 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46eaa689-3300-41bd-9c3d-cb995f2a5136 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:08:19.678645 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-46eaa689-3300-41bd-9c3d-cb995f2a5136 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:08:19.798389 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46eaa689-3300-41bd-9c3d-cb995f2a5136 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:08:19.800068 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46eaa689-3300-41bd-9c3d-cb995f2a5136 service neutron] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:08:19.820502 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 651/1314] 10.222.0.23 () {60 vars in 1365 bytes} [Thu May 19 12:08:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:08:22.687209 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb2e896-3e66-4fe2-87ff-f31b1ce7a48f None None] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:08:22.688106 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb2e896-3e66-4fe2-87ff-f31b1ce7a48f None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:08:22.688248 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb2e896-3e66-4fe2-87ff-f31b1ce7a48f None None] PATH_INFO: `/` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:08:22.689636 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 664/1315] 10.222.0.23 () {54 vars in 926 bytes} [Thu May 19 12:08:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 12:08:22.695123 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae83877-c009-4a64-b909-09fd71e30420 None None] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:08:22.695610 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae83877-c009-4a64-b909-09fd71e30420 None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:08:22.695715 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae83877-c009-4a64-b909-09fd71e30420 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:08:22.760598 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.auth.core [None req-4ae83877-c009-4a64-b909-09fd71e30420 None None] MFA Rules not processed for user `267c35a521bb473ea2784646137bebda`. Rule list: `[]` (Enabled: `True`). {{(pid=80539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:08:22.761847 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-4ae83877-c009-4a64-b909-09fd71e30420 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:08:22.813452 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:08:22.858770 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 652/1316] 10.222.0.23 () {58 vars in 1023 bytes} [Thu May 19 12:08:22 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:08:22.914652 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2167741f-15f4-4396-a5ec-e436b9f3fa8c None None] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:08:22.914942 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2167741f-15f4-4396-a5ec-e436b9f3fa8c None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:08:22.915219 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2167741f-15f4-4396-a5ec-e436b9f3fa8c None None] PATH_INFO: `/` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:08:22.916630 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 665/1317] 10.222.0.23 () {54 vars in 964 bytes} [Thu May 19 12:08:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 12:08:22.923768 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ee5abf-e929-413a-b4d0-164d38962022 None None] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:08:22.924099 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ee5abf-e929-413a-b4d0-164d38962022 None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:08:22.924405 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ee5abf-e929-413a-b4d0-164d38962022 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:08:23.002926 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.auth.core [None req-83ee5abf-e929-413a-b4d0-164d38962022 None None] MFA Rules not processed for user `1a00b3fa7497456895e5e0bf105c4fc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:08:23.004183 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-83ee5abf-e929-413a-b4d0-164d38962022 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:08:23.085710 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 653/1318] 10.222.0.23 () {58 vars in 1061 bytes} [Thu May 19 12:08:22 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:08:23.095274 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2167741f-15f4-4396-a5ec-e436b9f3fa8c None None] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:08:23.096863 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-2167741f-15f4-4396-a5ec-e436b9f3fa8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:08:23.107006 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:08:23.211141 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2167741f-15f4-4396-a5ec-e436b9f3fa8c None None] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:08:23.287662 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04584aa-a380-4656-bf36-9049101e53c9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a00b3fa7497456895e5e0bf105c4fc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd70d25bf69314820b8aaecb65576978a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:08:23.288336 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04584aa-a380-4656-bf36-9049101e53c9 service neutron] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:08:23.288623 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04584aa-a380-4656-bf36-9049101e53c9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:08:23.288900 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04584aa-a380-4656-bf36-9049101e53c9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:08:23.290563 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-c04584aa-a380-4656-bf36-9049101e53c9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:08:23.352716 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:08:23.399178 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04584aa-a380-4656-bf36-9049101e53c9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:08:23.400664 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04584aa-a380-4656-bf36-9049101e53c9 service neutron] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:08:23.419646 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 666/1319] 10.222.0.23 () {60 vars in 1365 bytes} [Thu May 19 12:08:23 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:08:26.957233 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008b349a-1702-4d76-a5eb-c80376a08517 None None] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:08:26.957919 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008b349a-1702-4d76-a5eb-c80376a08517 None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:08:26.957919 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008b349a-1702-4d76-a5eb-c80376a08517 None None] PATH_INFO: `/` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:08:26.959173 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 654/1320] 10.222.0.23 () {54 vars in 926 bytes} [Thu May 19 12:08:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 12:08:26.965144 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130d7bfc-01d9-4566-b95c-a2facdb6359f None None] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:08:26.965582 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130d7bfc-01d9-4566-b95c-a2facdb6359f None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:08:26.965722 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130d7bfc-01d9-4566-b95c-a2facdb6359f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:08:27.029075 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.auth.core [None req-130d7bfc-01d9-4566-b95c-a2facdb6359f None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:08:27.030273 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-130d7bfc-01d9-4566-b95c-a2facdb6359f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:08:27.137839 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 667/1321] 10.222.0.23 () {58 vars in 1023 bytes} [Thu May 19 12:08:26 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:08:27.181118 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008b349a-1702-4d76-a5eb-c80376a08517 None None] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:08:27.182652 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-008b349a-1702-4d76-a5eb-c80376a08517 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:08:27.233703 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:08:27.287686 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008b349a-1702-4d76-a5eb-c80376a08517 None None] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:08:27.365553 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94a318b-5f13-48b9-92e6-7350a9b487d7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a00b3fa7497456895e5e0bf105c4fc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd70d25bf69314820b8aaecb65576978a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:08:27.366201 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94a318b-5f13-48b9-92e6-7350a9b487d7 service neutron] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:08:27.366484 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94a318b-5f13-48b9-92e6-7350a9b487d7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:08:27.366756 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94a318b-5f13-48b9-92e6-7350a9b487d7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:08:27.368711 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-d94a318b-5f13-48b9-92e6-7350a9b487d7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:08:27.472180 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:08:27.485953 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94a318b-5f13-48b9-92e6-7350a9b487d7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:08:27.487530 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94a318b-5f13-48b9-92e6-7350a9b487d7 service neutron] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:08:27.509052 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 655/1322] 10.222.0.23 () {60 vars in 1365 bytes} [Thu May 19 12:08:27 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:08:31.139023 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac22fc81-a6f2-4cec-a7dc-2d99c19b865b None None] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:08:31.139588 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac22fc81-a6f2-4cec-a7dc-2d99c19b865b None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:08:31.139588 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac22fc81-a6f2-4cec-a7dc-2d99c19b865b None None] PATH_INFO: `/` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:08:31.141500 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 668/1323] 10.222.0.23 () {54 vars in 926 bytes} [Thu May 19 12:08:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 12:08:31.147668 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e3665a-b588-4b1c-ad32-214a90b67c23 None None] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:08:31.148144 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e3665a-b588-4b1c-ad32-214a90b67c23 None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:08:31.148248 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e3665a-b588-4b1c-ad32-214a90b67c23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:08:31.224453 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.auth.core [None req-a1e3665a-b588-4b1c-ad32-214a90b67c23 None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:08:31.226429 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-a1e3665a-b588-4b1c-ad32-214a90b67c23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:08:31.340371 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 656/1324] 10.222.0.23 () {58 vars in 1023 bytes} [Thu May 19 12:08:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:08:31.376983 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac22fc81-a6f2-4cec-a7dc-2d99c19b865b None None] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:08:31.378639 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-ac22fc81-a6f2-4cec-a7dc-2d99c19b865b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:08:31.399158 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:08:31.489237 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac22fc81-a6f2-4cec-a7dc-2d99c19b865b None None] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:08:31.579691 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc93a11-8589-4cb3-9a99-348493f11c9a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a00b3fa7497456895e5e0bf105c4fc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd70d25bf69314820b8aaecb65576978a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:08:31.580387 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc93a11-8589-4cb3-9a99-348493f11c9a service neutron] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:08:31.580672 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc93a11-8589-4cb3-9a99-348493f11c9a service neutron] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:08:31.580951 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc93a11-8589-4cb3-9a99-348493f11c9a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:08:31.582718 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-6fc93a11-8589-4cb3-9a99-348493f11c9a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:08:31.666522 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:08:31.708820 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc93a11-8589-4cb3-9a99-348493f11c9a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:08:31.710547 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc93a11-8589-4cb3-9a99-348493f11c9a service neutron] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:08:31.729893 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 669/1325] 10.222.0.23 () {60 vars in 1365 bytes} [Thu May 19 12:08:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:08:34.066976 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7262e8-97c7-4b74-a8a3-d3691d05ec7f None None] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:08:34.067537 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7262e8-97c7-4b74-a8a3-d3691d05ec7f None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:08:34.067537 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7262e8-97c7-4b74-a8a3-d3691d05ec7f None None] PATH_INFO: `/` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:08:34.068940 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 657/1326] 10.222.0.23 () {54 vars in 926 bytes} [Thu May 19 12:08:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 12:08:34.074470 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f21dacd-2161-4966-adff-e1d0ff6affb7 None None] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:08:34.074921 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f21dacd-2161-4966-adff-e1d0ff6affb7 None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:08:34.075043 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f21dacd-2161-4966-adff-e1d0ff6affb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:08:34.144414 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.auth.core [None req-3f21dacd-2161-4966-adff-e1d0ff6affb7 None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:08:34.145547 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-3f21dacd-2161-4966-adff-e1d0ff6affb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:08:34.249566 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 670/1327] 10.222.0.23 () {58 vars in 1023 bytes} [Thu May 19 12:08:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:08:34.286394 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7262e8-97c7-4b74-a8a3-d3691d05ec7f None None] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:08:34.288015 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-aa7262e8-97c7-4b74-a8a3-d3691d05ec7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:08:34.354088 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:08:34.401544 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7262e8-97c7-4b74-a8a3-d3691d05ec7f None None] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:08:34.483564 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21180646-e80d-4bc2-a51e-dce2f66f0bcf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a00b3fa7497456895e5e0bf105c4fc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd70d25bf69314820b8aaecb65576978a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:08:34.483995 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21180646-e80d-4bc2-a51e-dce2f66f0bcf service neutron] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:08:34.484255 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21180646-e80d-4bc2-a51e-dce2f66f0bcf service neutron] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:08:34.485024 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21180646-e80d-4bc2-a51e-dce2f66f0bcf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:08:34.487331 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-21180646-e80d-4bc2-a51e-dce2f66f0bcf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:08:34.610449 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:08:34.613583 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21180646-e80d-4bc2-a51e-dce2f66f0bcf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:08:34.615108 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21180646-e80d-4bc2-a51e-dce2f66f0bcf service neutron] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:08:34.636344 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 658/1328] 10.222.0.23 () {60 vars in 1365 bytes} [Thu May 19 12:08:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:08:36.984003 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0282d99-8e23-4235-936f-a7867470fb68 None None] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:08:36.984589 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0282d99-8e23-4235-936f-a7867470fb68 None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:08:36.984589 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0282d99-8e23-4235-936f-a7867470fb68 None None] PATH_INFO: `/` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:08:36.986672 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 671/1329] 10.222.0.23 () {54 vars in 926 bytes} [Thu May 19 12:08:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 12:08:36.992322 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7781e34-a8fa-43e5-a6db-280fdfdae23c None None] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:08:36.992764 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7781e34-a8fa-43e5-a6db-280fdfdae23c None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:08:36.992879 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7781e34-a8fa-43e5-a6db-280fdfdae23c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:08:37.060233 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.auth.core [None req-a7781e34-a8fa-43e5-a6db-280fdfdae23c None None] MFA Rules not processed for user `267c35a521bb473ea2784646137bebda`. Rule list: `[]` (Enabled: `True`). {{(pid=80539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:08:37.061479 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-a7781e34-a8fa-43e5-a6db-280fdfdae23c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:08:37.168270 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 659/1330] 10.222.0.23 () {58 vars in 1023 bytes} [Thu May 19 12:08:36 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:08:37.208162 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0282d99-8e23-4235-936f-a7867470fb68 None None] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:08:37.209792 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-b0282d99-8e23-4235-936f-a7867470fb68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:08:37.242860 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:08:37.323059 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0282d99-8e23-4235-936f-a7867470fb68 None None] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:08:37.401433 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b05aa26-4991-44eb-b4f9-89f9398ba6f3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a00b3fa7497456895e5e0bf105c4fc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd70d25bf69314820b8aaecb65576978a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:08:37.402092 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b05aa26-4991-44eb-b4f9-89f9398ba6f3 service neutron] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:08:37.402480 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b05aa26-4991-44eb-b4f9-89f9398ba6f3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:08:37.402844 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b05aa26-4991-44eb-b4f9-89f9398ba6f3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:08:37.404925 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-6b05aa26-4991-44eb-b4f9-89f9398ba6f3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:08:37.485560 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:08:37.520383 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b05aa26-4991-44eb-b4f9-89f9398ba6f3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:08:37.521842 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b05aa26-4991-44eb-b4f9-89f9398ba6f3 service neutron] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:08:37.540144 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 672/1331] 10.222.0.23 () {60 vars in 1365 bytes} [Thu May 19 12:08:37 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:08:40.376413 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf2d0aa-a58b-42fa-8d83-2d43eb794ce3 None None] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:08:40.376954 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf2d0aa-a58b-42fa-8d83-2d43eb794ce3 None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:08:40.376954 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf2d0aa-a58b-42fa-8d83-2d43eb794ce3 None None] PATH_INFO: `/` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:08:40.378418 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 660/1332] 10.222.0.23 () {54 vars in 926 bytes} [Thu May 19 12:08:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 12:08:40.384147 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4ff22c-a04c-41a3-b8bd-de4d3ba03cf6 None None] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:08:40.384588 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4ff22c-a04c-41a3-b8bd-de4d3ba03cf6 None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:08:40.384719 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4ff22c-a04c-41a3-b8bd-de4d3ba03cf6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:08:40.457088 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.auth.core [None req-7a4ff22c-a04c-41a3-b8bd-de4d3ba03cf6 None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:08:40.458341 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-7a4ff22c-a04c-41a3-b8bd-de4d3ba03cf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:08:40.559199 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 673/1333] 10.222.0.23 () {58 vars in 1023 bytes} [Thu May 19 12:08:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:08:40.596716 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf2d0aa-a58b-42fa-8d83-2d43eb794ce3 None None] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:08:40.598249 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-aaf2d0aa-a58b-42fa-8d83-2d43eb794ce3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:08:40.680045 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:08:40.718456 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf2d0aa-a58b-42fa-8d83-2d43eb794ce3 None None] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:08:40.800413 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12e7b33-9bdf-44a2-8931-86fbc9c1c3d1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a00b3fa7497456895e5e0bf105c4fc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd70d25bf69314820b8aaecb65576978a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:08:40.801079 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12e7b33-9bdf-44a2-8931-86fbc9c1c3d1 service neutron] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:08:40.801367 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12e7b33-9bdf-44a2-8931-86fbc9c1c3d1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:08:40.801639 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12e7b33-9bdf-44a2-8931-86fbc9c1c3d1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:08:40.803363 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-b12e7b33-9bdf-44a2-8931-86fbc9c1c3d1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:08:40.940075 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12e7b33-9bdf-44a2-8931-86fbc9c1c3d1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:08:40.941548 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12e7b33-9bdf-44a2-8931-86fbc9c1c3d1 service neutron] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:08:40.955332 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:08:40.965100 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 661/1334] 10.222.0.23 () {60 vars in 1365 bytes} [Thu May 19 12:08:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:08:43.318218 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4861995a-3a3d-4586-95a2-c936d0f8e479 None None] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:08:43.318777 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4861995a-3a3d-4586-95a2-c936d0f8e479 None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:08:43.318875 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4861995a-3a3d-4586-95a2-c936d0f8e479 None None] PATH_INFO: `/` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:08:43.320316 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 674/1335] 10.222.0.23 () {54 vars in 926 bytes} [Thu May 19 12:08:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 12:08:43.325659 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4372a98-47e6-4315-b779-66bea9abe358 None None] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:08:43.326221 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4372a98-47e6-4315-b779-66bea9abe358 None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:08:43.326221 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4372a98-47e6-4315-b779-66bea9abe358 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:08:43.392498 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.auth.core [None req-e4372a98-47e6-4315-b779-66bea9abe358 None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:08:43.393727 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-e4372a98-47e6-4315-b779-66bea9abe358 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:08:43.491761 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 662/1336] 10.222.0.23 () {58 vars in 1023 bytes} [Thu May 19 12:08:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:08:43.532977 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4861995a-3a3d-4586-95a2-c936d0f8e479 None None] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:08:43.534625 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-4861995a-3a3d-4586-95a2-c936d0f8e479 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:08:43.575237 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:08:43.642075 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4861995a-3a3d-4586-95a2-c936d0f8e479 None None] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:08:43.716340 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eebf57f-28b5-4390-895a-71e6c5472bbb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a00b3fa7497456895e5e0bf105c4fc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd70d25bf69314820b8aaecb65576978a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:08:43.716998 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eebf57f-28b5-4390-895a-71e6c5472bbb service neutron] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:08:43.717287 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eebf57f-28b5-4390-895a-71e6c5472bbb service neutron] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:08:43.717566 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eebf57f-28b5-4390-895a-71e6c5472bbb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:08:43.719862 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-7eebf57f-28b5-4390-895a-71e6c5472bbb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:08:43.805751 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:08:43.832937 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eebf57f-28b5-4390-895a-71e6c5472bbb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:08:43.834807 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eebf57f-28b5-4390-895a-71e6c5472bbb service neutron] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:08:43.856480 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 675/1337] 10.222.0.23 () {60 vars in 1365 bytes} [Thu May 19 12:08:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:08:46.817562 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e67157-f265-4cf9-83ab-8182ddb14b0a None None] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:08:46.818149 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e67157-f265-4cf9-83ab-8182ddb14b0a None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:08:46.818149 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e67157-f265-4cf9-83ab-8182ddb14b0a None None] PATH_INFO: `/` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:08:46.819705 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 663/1338] 10.222.0.23 () {54 vars in 926 bytes} [Thu May 19 12:08:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 12:08:46.826843 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1e5d8f-011a-44d2-8bbf-d9cf2f8d0c61 None None] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:08:46.827318 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1e5d8f-011a-44d2-8bbf-d9cf2f8d0c61 None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:08:46.827433 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1e5d8f-011a-44d2-8bbf-d9cf2f8d0c61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:08:46.900906 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.auth.core [None req-fa1e5d8f-011a-44d2-8bbf-d9cf2f8d0c61 None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:08:46.902128 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-fa1e5d8f-011a-44d2-8bbf-d9cf2f8d0c61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:08:47.011632 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 676/1339] 10.222.0.23 () {58 vars in 1023 bytes} [Thu May 19 12:08:46 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:08:47.048593 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e67157-f265-4cf9-83ab-8182ddb14b0a None None] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:08:47.050765 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-83e67157-f265-4cf9-83ab-8182ddb14b0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:08:47.133480 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:08:47.149249 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e67157-f265-4cf9-83ab-8182ddb14b0a None None] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:08:47.154558 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55445d8-4f0c-4a27-85dc-dd5e1834bd83 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a00b3fa7497456895e5e0bf105c4fc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd70d25bf69314820b8aaecb65576978a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:08:47.155209 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55445d8-4f0c-4a27-85dc-dd5e1834bd83 service neutron] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:08:47.155494 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55445d8-4f0c-4a27-85dc-dd5e1834bd83 service neutron] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:08:47.155772 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55445d8-4f0c-4a27-85dc-dd5e1834bd83 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:08:47.171146 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55445d8-4f0c-4a27-85dc-dd5e1834bd83 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:08:47.173158 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55445d8-4f0c-4a27-85dc-dd5e1834bd83 service neutron] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:08:47.183178 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 664/1340] 10.222.0.23 () {60 vars in 1365 bytes} [Thu May 19 12:08:47 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:08:51.840537 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be560075-3e95-44a2-9197-14c237203cdf None None] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:08:51.841126 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be560075-3e95-44a2-9197-14c237203cdf None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:08:51.841126 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be560075-3e95-44a2-9197-14c237203cdf None None] PATH_INFO: `/` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:08:51.842492 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 677/1341] 10.222.0.23 () {54 vars in 926 bytes} [Thu May 19 12:08:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 12:08:51.848082 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64fa008-74ba-456c-ab30-4aceda78bd76 None None] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:08:51.848551 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64fa008-74ba-456c-ab30-4aceda78bd76 None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:08:51.848669 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64fa008-74ba-456c-ab30-4aceda78bd76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:08:51.879975 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.auth.core [None req-c64fa008-74ba-456c-ab30-4aceda78bd76 None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:08:51.881221 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-c64fa008-74ba-456c-ab30-4aceda78bd76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:08:51.962533 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 665/1342] 10.222.0.23 () {58 vars in 1023 bytes} [Thu May 19 12:08:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:08:52.001150 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be560075-3e95-44a2-9197-14c237203cdf None None] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:08:52.011946 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be560075-3e95-44a2-9197-14c237203cdf None None] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:08:52.016279 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4703d1a5-1811-4557-8530-45cf7e03567f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a00b3fa7497456895e5e0bf105c4fc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd70d25bf69314820b8aaecb65576978a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:08:52.017139 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4703d1a5-1811-4557-8530-45cf7e03567f service neutron] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:08:52.017547 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4703d1a5-1811-4557-8530-45cf7e03567f service neutron] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:08:52.017846 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4703d1a5-1811-4557-8530-45cf7e03567f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:08:52.027367 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4703d1a5-1811-4557-8530-45cf7e03567f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:08:52.028950 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4703d1a5-1811-4557-8530-45cf7e03567f service neutron] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:08:52.039403 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 678/1343] 10.222.0.23 () {60 vars in 1365 bytes} [Thu May 19 12:08:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:08:55.338818 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfdfc4c-f5ee-4029-8648-8fb9d813c26a None None] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:08:55.339401 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfdfc4c-f5ee-4029-8648-8fb9d813c26a None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:08:55.339401 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfdfc4c-f5ee-4029-8648-8fb9d813c26a None None] PATH_INFO: `/` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:08:55.340770 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 666/1344] 10.222.0.23 () {54 vars in 926 bytes} [Thu May 19 12:08:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 12:08:55.346295 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e997b4af-bb6a-455c-973c-04c89917af20 None None] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:08:55.346760 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e997b4af-bb6a-455c-973c-04c89917af20 None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:08:55.346991 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e997b4af-bb6a-455c-973c-04c89917af20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:08:55.365865 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.auth.core [None req-e997b4af-bb6a-455c-973c-04c89917af20 None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:08:55.367079 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-e997b4af-bb6a-455c-973c-04c89917af20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:08:55.375863 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 679/1345] 10.222.0.23 () {58 vars in 1023 bytes} [Thu May 19 12:08:55 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:08:55.412062 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfdfc4c-f5ee-4029-8648-8fb9d813c26a None None] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:08:55.423381 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfdfc4c-f5ee-4029-8648-8fb9d813c26a None None] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:08:55.426907 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb5015b-70e7-4cd0-acc9-555c815accee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a00b3fa7497456895e5e0bf105c4fc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd70d25bf69314820b8aaecb65576978a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:08:55.427545 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb5015b-70e7-4cd0-acc9-555c815accee service neutron] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:08:55.427864 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb5015b-70e7-4cd0-acc9-555c815accee service neutron] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:08:55.428146 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb5015b-70e7-4cd0-acc9-555c815accee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:08:55.438051 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb5015b-70e7-4cd0-acc9-555c815accee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:08:55.439479 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb5015b-70e7-4cd0-acc9-555c815accee service neutron] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:08:55.449719 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 667/1346] 10.222.0.23 () {60 vars in 1365 bytes} [Thu May 19 12:08:55 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:09:00.049799 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2ee8b3-eed7-4319-9311-b96a14d84333 None None] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:09:00.050326 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2ee8b3-eed7-4319-9311-b96a14d84333 None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:09:00.050434 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2ee8b3-eed7-4319-9311-b96a14d84333 None None] PATH_INFO: `/` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:09:00.051962 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 680/1347] 10.222.0.23 () {54 vars in 926 bytes} [Thu May 19 12:09:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 12:09:00.057552 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330b4f4a-efa8-4b0b-8108-0d7b9af97fa3 None None] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:09:00.058087 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330b4f4a-efa8-4b0b-8108-0d7b9af97fa3 None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:09:00.058087 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330b4f4a-efa8-4b0b-8108-0d7b9af97fa3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:09:00.077274 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.auth.core [None req-330b4f4a-efa8-4b0b-8108-0d7b9af97fa3 None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:09:00.078579 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-330b4f4a-efa8-4b0b-8108-0d7b9af97fa3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:09:00.087144 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 668/1348] 10.222.0.23 () {58 vars in 1023 bytes} [Thu May 19 12:09:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:09:00.125256 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2ee8b3-eed7-4319-9311-b96a14d84333 None None] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:09:00.137174 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2ee8b3-eed7-4319-9311-b96a14d84333 None None] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:09:00.141448 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12743053-e79b-43c6-b236-995e121e6a2a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a00b3fa7497456895e5e0bf105c4fc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd70d25bf69314820b8aaecb65576978a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:09:00.142115 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12743053-e79b-43c6-b236-995e121e6a2a service neutron] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:09:00.145165 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12743053-e79b-43c6-b236-995e121e6a2a service neutron] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:09:00.145165 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12743053-e79b-43c6-b236-995e121e6a2a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:09:00.152796 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12743053-e79b-43c6-b236-995e121e6a2a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:09:00.154310 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12743053-e79b-43c6-b236-995e121e6a2a service neutron] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:09:00.167154 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 681/1349] 10.222.0.23 () {60 vars in 1365 bytes} [Thu May 19 12:09:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:09:05.795746 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3730fd-af1f-4c5a-8a68-38fa20c817e5 None None] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:09:05.797289 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3730fd-af1f-4c5a-8a68-38fa20c817e5 None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:09:05.797794 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3730fd-af1f-4c5a-8a68-38fa20c817e5 None None] PATH_INFO: `/` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:09:05.800056 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 669/1350] 10.222.0.23 () {54 vars in 921 bytes} [Thu May 19 12:09:05 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 12:09:05.805408 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade9da8d-091b-4fc5-bc2a-b5462abf1aff None None] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:09:05.806249 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade9da8d-091b-4fc5-bc2a-b5462abf1aff None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:09:05.806760 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade9da8d-091b-4fc5-bc2a-b5462abf1aff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:09:05.822418 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:09:05.889241 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.auth.core [None req-ade9da8d-091b-4fc5-bc2a-b5462abf1aff None None] MFA Rules not processed for user `50f937eed4f3414fa6f19cab2a84eb5f`. Rule list: `[]` (Enabled: `True`). {{(pid=80540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:09:05.891351 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-ade9da8d-091b-4fc5-bc2a-b5462abf1aff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:09:05.984623 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 682/1351] 10.222.0.23 () {58 vars in 1018 bytes} [Thu May 19 12:09:05 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 12:09:10.071499 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=57 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:09:10.638944 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9c15fb-c9df-48e9-a8b3-742e22a37456 None None] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:09:10.639410 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9c15fb-c9df-48e9-a8b3-742e22a37456 None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:09:10.639539 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9c15fb-c9df-48e9-a8b3-742e22a37456 None None] PATH_INFO: `/` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:09:10.641883 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 670/1352] 10.222.0.23 () {54 vars in 921 bytes} [Thu May 19 12:09:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 12:09:10.647661 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c790c3ee-bb74-4f0e-a9de-9d058dfa394d None None] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:09:10.648222 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c790c3ee-bb74-4f0e-a9de-9d058dfa394d None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:09:10.648462 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c790c3ee-bb74-4f0e-a9de-9d058dfa394d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:09:10.668649 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.auth.core [None req-c790c3ee-bb74-4f0e-a9de-9d058dfa394d None None] MFA Rules not processed for user `50f937eed4f3414fa6f19cab2a84eb5f`. Rule list: `[]` (Enabled: `True`). {{(pid=80540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:09:10.669868 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-c790c3ee-bb74-4f0e-a9de-9d058dfa394d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:09:10.679082 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 683/1353] 10.222.0.23 () {58 vars in 1018 bytes} [Thu May 19 12:09:10 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:09:12.211557 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1060b097-e995-4c8a-a0c9-a6b3227eb6bf None None] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:09:12.212191 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1060b097-e995-4c8a-a0c9-a6b3227eb6bf None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:09:12.212191 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1060b097-e995-4c8a-a0c9-a6b3227eb6bf None None] PATH_INFO: `/` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:09:12.213400 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 671/1354] 10.222.0.23 () {54 vars in 921 bytes} [Thu May 19 12:09:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 12:09:12.218889 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6780e9b-6c19-4ec1-9753-8eafee7b702e None None] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:09:12.219610 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6780e9b-6c19-4ec1-9753-8eafee7b702e None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:09:12.219883 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6780e9b-6c19-4ec1-9753-8eafee7b702e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:09:12.239956 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.auth.core [None req-d6780e9b-6c19-4ec1-9753-8eafee7b702e None None] MFA Rules not processed for user `50f937eed4f3414fa6f19cab2a84eb5f`. Rule list: `[]` (Enabled: `True`). {{(pid=80540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:09:12.241197 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-d6780e9b-6c19-4ec1-9753-8eafee7b702e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:09:12.251493 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 684/1355] 10.222.0.23 () {58 vars in 1018 bytes} [Thu May 19 12:09:12 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:09:16.226129 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e66298-c64a-451b-bb9f-c7cca9c0fd2f None None] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:09:16.226687 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e66298-c64a-451b-bb9f-c7cca9c0fd2f None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:09:16.226687 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e66298-c64a-451b-bb9f-c7cca9c0fd2f None None] PATH_INFO: `/` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:09:16.228164 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 672/1356] 10.222.0.23 () {54 vars in 926 bytes} [Thu May 19 12:09:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 12:09:16.233591 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091b1246-3372-46e1-824f-87f38b348c9b None None] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:09:16.234801 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091b1246-3372-46e1-824f-87f38b348c9b None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:09:16.234801 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091b1246-3372-46e1-824f-87f38b348c9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:09:16.254727 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.auth.core [None req-091b1246-3372-46e1-824f-87f38b348c9b None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:09:16.256043 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-091b1246-3372-46e1-824f-87f38b348c9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:09:16.264480 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 685/1357] 10.222.0.23 () {58 vars in 1023 bytes} [Thu May 19 12:09:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:09:16.324369 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f74b60-048f-434f-8b18-83e6f39a536e None None] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:09:16.324716 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f74b60-048f-434f-8b18-83e6f39a536e None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:09:16.324931 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f74b60-048f-434f-8b18-83e6f39a536e None None] PATH_INFO: `/` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:09:16.326569 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 673/1358] 10.222.0.23 () {54 vars in 961 bytes} [Thu May 19 12:09:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 12:09:16.340845 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16528ba-3776-4a75-baee-1b0acdffd64a None None] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:09:16.341947 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16528ba-3776-4a75-baee-1b0acdffd64a None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:09:16.342482 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16528ba-3776-4a75-baee-1b0acdffd64a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:09:16.549241 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.auth.core [None req-b16528ba-3776-4a75-baee-1b0acdffd64a None None] MFA Rules not processed for user `34a98d75d84b4c67981377e0f26eca62`. Rule list: `[]` (Enabled: `True`). {{(pid=80540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:09:16.550374 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-b16528ba-3776-4a75-baee-1b0acdffd64a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:09:16.630618 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:09:16.658731 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 686/1359] 10.222.0.23 () {58 vars in 1058 bytes} [Thu May 19 12:09:16 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 319 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:09:16.667701 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f74b60-048f-434f-8b18-83e6f39a536e None None] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:09:16.685258 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f74b60-048f-434f-8b18-83e6f39a536e None None] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:09:16.692337 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96edd28e-75c3-4236-9226-ac391de89555 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34a98d75d84b4c67981377e0f26eca62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd70d25bf69314820b8aaecb65576978a', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:09:16.693033 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96edd28e-75c3-4236-9226-ac391de89555 service nova] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:09:16.693329 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96edd28e-75c3-4236-9226-ac391de89555 service nova] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:09:16.693767 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96edd28e-75c3-4236-9226-ac391de89555 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:09:16.704380 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96edd28e-75c3-4236-9226-ac391de89555 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:09:16.706012 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96edd28e-75c3-4236-9226-ac391de89555 service nova] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:09:16.717888 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 674/1360] 10.222.0.23 () {60 vars in 1365 bytes} [Thu May 19 12:09:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:09:19.276911 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e09b496-9f59-4a26-a83b-03d915b1a091 None None] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:09:19.277654 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e09b496-9f59-4a26-a83b-03d915b1a091 None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:09:19.277654 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e09b496-9f59-4a26-a83b-03d915b1a091 None None] PATH_INFO: `/` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:09:19.279095 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 687/1361] 10.222.0.23 () {54 vars in 926 bytes} [Thu May 19 12:09:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 12:09:19.285346 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077813aa-a3f0-4c8a-9dbb-0c224c9602ed None None] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:09:19.285884 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077813aa-a3f0-4c8a-9dbb-0c224c9602ed None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:09:19.286001 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077813aa-a3f0-4c8a-9dbb-0c224c9602ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:09:19.305577 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.auth.core [None req-077813aa-a3f0-4c8a-9dbb-0c224c9602ed None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:09:19.306769 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-077813aa-a3f0-4c8a-9dbb-0c224c9602ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:09:19.321625 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 675/1362] 10.222.0.23 () {58 vars in 1023 bytes} [Thu May 19 12:09:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:09:19.373318 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e09b496-9f59-4a26-a83b-03d915b1a091 None None] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:09:19.388197 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e09b496-9f59-4a26-a83b-03d915b1a091 None None] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:09:19.395372 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e42b4ee-5510-47b9-9a40-cc82b992747b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34a98d75d84b4c67981377e0f26eca62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd70d25bf69314820b8aaecb65576978a', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:09:19.396133 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e42b4ee-5510-47b9-9a40-cc82b992747b service nova] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:09:19.396611 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e42b4ee-5510-47b9-9a40-cc82b992747b service nova] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:09:19.396910 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e42b4ee-5510-47b9-9a40-cc82b992747b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:09:19.407684 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e42b4ee-5510-47b9-9a40-cc82b992747b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:09:19.410253 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e42b4ee-5510-47b9-9a40-cc82b992747b service nova] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:09:19.421976 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 688/1363] 10.222.0.23 () {60 vars in 1365 bytes} [Thu May 19 12:09:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:09:22.306026 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d860361d-b47d-463b-a9ff-f8c9af8a7804 None None] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:09:22.306576 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d860361d-b47d-463b-a9ff-f8c9af8a7804 None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:09:22.306576 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d860361d-b47d-463b-a9ff-f8c9af8a7804 None None] PATH_INFO: `/` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:09:22.308016 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 676/1364] 10.222.0.23 () {54 vars in 926 bytes} [Thu May 19 12:09:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 12:09:22.314966 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658dfa73-36dc-46cf-8326-d6c317b7a692 None None] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:09:22.316007 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658dfa73-36dc-46cf-8326-d6c317b7a692 None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:09:22.316500 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658dfa73-36dc-46cf-8326-d6c317b7a692 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:09:22.336452 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.auth.core [None req-658dfa73-36dc-46cf-8326-d6c317b7a692 None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:09:22.337682 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-658dfa73-36dc-46cf-8326-d6c317b7a692 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:09:22.346441 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 689/1365] 10.222.0.23 () {58 vars in 1023 bytes} [Thu May 19 12:09:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:09:22.393632 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a6e84c-7034-4a81-88a7-ab1ff1183407 None None] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:09:22.393864 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a6e84c-7034-4a81-88a7-ab1ff1183407 None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:09:22.394031 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a6e84c-7034-4a81-88a7-ab1ff1183407 None None] PATH_INFO: `/` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:09:22.395599 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 677/1366] 10.222.0.23 () {54 vars in 961 bytes} [Thu May 19 12:09:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 12:09:22.401219 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088b0cc5-f91d-467d-96ae-9994409f5b88 None None] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:09:22.401582 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088b0cc5-f91d-467d-96ae-9994409f5b88 None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:09:22.401927 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088b0cc5-f91d-467d-96ae-9994409f5b88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:09:22.421819 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.auth.core [None req-088b0cc5-f91d-467d-96ae-9994409f5b88 None None] MFA Rules not processed for user `34a98d75d84b4c67981377e0f26eca62`. Rule list: `[]` (Enabled: `True`). {{(pid=80540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:09:22.423075 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-088b0cc5-f91d-467d-96ae-9994409f5b88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:09:22.433599 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 690/1367] 10.222.0.23 () {58 vars in 1058 bytes} [Thu May 19 12:09:22 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:09:22.442583 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a6e84c-7034-4a81-88a7-ab1ff1183407 None None] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:09:22.458492 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a6e84c-7034-4a81-88a7-ab1ff1183407 None None] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:09:22.464626 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc8786e-a7ce-4800-817d-03ce77d67fa0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34a98d75d84b4c67981377e0f26eca62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd70d25bf69314820b8aaecb65576978a', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:09:22.465296 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc8786e-a7ce-4800-817d-03ce77d67fa0 service nova] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:09:22.465577 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc8786e-a7ce-4800-817d-03ce77d67fa0 service nova] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:09:22.465852 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc8786e-a7ce-4800-817d-03ce77d67fa0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:09:22.475398 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc8786e-a7ce-4800-817d-03ce77d67fa0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:09:22.477061 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc8786e-a7ce-4800-817d-03ce77d67fa0 service nova] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:09:22.488233 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 678/1368] 10.222.0.23 () {60 vars in 1365 bytes} [Thu May 19 12:09:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:09:25.103030 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373e89b5-099c-4b62-9681-3d564decf1de None None] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:09:25.103571 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373e89b5-099c-4b62-9681-3d564decf1de None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:09:25.103571 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373e89b5-099c-4b62-9681-3d564decf1de None None] PATH_INFO: `/` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:09:25.103914 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f68154-f9bc-4840-9cba-b074c717b8b4 None None] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:09:25.104691 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f68154-f9bc-4840-9cba-b074c717b8b4 None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:09:25.105059 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 691/1369] 10.222.0.23 () {54 vars in 926 bytes} [Thu May 19 12:09:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 12:09:25.105448 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f68154-f9bc-4840-9cba-b074c717b8b4 None None] PATH_INFO: `/` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:09:25.107151 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 679/1370] 10.222.0.23 () {54 vars in 926 bytes} [Thu May 19 12:09:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 12:09:25.111443 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5966bc8-fc60-4721-bb56-504641faf0e6 None None] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:09:25.111736 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5966bc8-fc60-4721-bb56-504641faf0e6 None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:09:25.111969 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5966bc8-fc60-4721-bb56-504641faf0e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:09:25.113997 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d678402-2828-405b-84df-6fb5c0f3b5d4 None None] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:09:25.114584 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d678402-2828-405b-84df-6fb5c0f3b5d4 None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:09:25.115284 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d678402-2828-405b-84df-6fb5c0f3b5d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:09:25.131890 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.auth.core [None req-f5966bc8-fc60-4721-bb56-504641faf0e6 None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:09:25.133515 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-f5966bc8-fc60-4721-bb56-504641faf0e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:09:25.140356 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.auth.core [None req-2d678402-2828-405b-84df-6fb5c0f3b5d4 None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:09:25.142440 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-2d678402-2828-405b-84df-6fb5c0f3b5d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:09:25.159887 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 680/1371] 10.222.0.23 () {58 vars in 1023 bytes} [Thu May 19 12:09:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:09:25.160202 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 692/1372] 10.222.0.23 () {58 vars in 1023 bytes} [Thu May 19 12:09:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:09:25.203917 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d678402-2828-405b-84df-6fb5c0f3b5d4 None None] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:09:25.221450 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d678402-2828-405b-84df-6fb5c0f3b5d4 None None] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:09:25.229257 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aad2265-24fd-4d7c-870a-e9ff52ddbdfe service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34a98d75d84b4c67981377e0f26eca62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd70d25bf69314820b8aaecb65576978a', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:09:25.229926 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aad2265-24fd-4d7c-870a-e9ff52ddbdfe service nova] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:09:25.230215 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aad2265-24fd-4d7c-870a-e9ff52ddbdfe service nova] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:09:25.230503 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aad2265-24fd-4d7c-870a-e9ff52ddbdfe service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:09:25.241936 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aad2265-24fd-4d7c-870a-e9ff52ddbdfe service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:09:25.243580 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aad2265-24fd-4d7c-870a-e9ff52ddbdfe service nova] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:09:25.255416 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 681/1373] 10.222.0.23 () {60 vars in 1365 bytes} [Thu May 19 12:09:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:09:25.464349 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88a4168-5721-45c3-8c08-9a8725ace701 None None] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:09:25.464638 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88a4168-5721-45c3-8c08-9a8725ace701 None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:09:25.465222 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88a4168-5721-45c3-8c08-9a8725ace701 None None] PATH_INFO: `/` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:09:25.466953 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 693/1374] 10.222.0.23 () {54 vars in 963 bytes} [Thu May 19 12:09:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 12:09:25.473818 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c56dd6-e87c-4cf1-9f81-e15224b31304 None None] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:09:25.474180 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c56dd6-e87c-4cf1-9f81-e15224b31304 None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:09:25.474391 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c56dd6-e87c-4cf1-9f81-e15224b31304 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:09:25.548818 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.auth.core [None req-70c56dd6-e87c-4cf1-9f81-e15224b31304 None None] MFA Rules not processed for user `8800254c0e784e42a791fb9abc274d79`. Rule list: `[]` (Enabled: `True`). {{(pid=80539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:09:25.550972 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-70c56dd6-e87c-4cf1-9f81-e15224b31304 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:09:25.647984 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 682/1375] 10.222.0.23 () {58 vars in 1060 bytes} [Thu May 19 12:09:25 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:09:25.657477 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88a4168-5721-45c3-8c08-9a8725ace701 None None] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:09:25.676284 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88a4168-5721-45c3-8c08-9a8725ace701 None None] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:09:25.680488 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba340b9-567a-439d-af4e-520342e8f85e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8800254c0e784e42a791fb9abc274d79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd70d25bf69314820b8aaecb65576978a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:09:25.681323 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba340b9-567a-439d-af4e-520342e8f85e service cinder] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:09:25.681786 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba340b9-567a-439d-af4e-520342e8f85e service cinder] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:09:25.682217 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba340b9-567a-439d-af4e-520342e8f85e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:09:25.696583 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba340b9-567a-439d-af4e-520342e8f85e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:09:25.698454 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba340b9-567a-439d-af4e-520342e8f85e service cinder] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:09:25.711843 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 694/1376] 10.222.0.23 () {60 vars in 1365 bytes} [Thu May 19 12:09:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:09:27.742000 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f88f48-b1f8-4af9-bb6f-bd4dbadef2f1 None None] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:09:27.742980 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f88f48-b1f8-4af9-bb6f-bd4dbadef2f1 None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:09:27.743420 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f88f48-b1f8-4af9-bb6f-bd4dbadef2f1 None None] PATH_INFO: `/` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:09:27.745150 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 683/1377] 10.222.0.23 () {54 vars in 926 bytes} [Thu May 19 12:09:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 12:09:27.751912 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38905c2-1acc-40cb-a4b4-ea8b7450008a None None] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:09:27.752506 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38905c2-1acc-40cb-a4b4-ea8b7450008a None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:09:27.752506 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38905c2-1acc-40cb-a4b4-ea8b7450008a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:09:27.775215 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.auth.core [None req-d38905c2-1acc-40cb-a4b4-ea8b7450008a None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:09:27.777727 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-d38905c2-1acc-40cb-a4b4-ea8b7450008a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:09:27.788162 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 695/1378] 10.222.0.23 () {58 vars in 1023 bytes} [Thu May 19 12:09:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:09:27.834144 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f88f48-b1f8-4af9-bb6f-bd4dbadef2f1 None None] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:09:27.850990 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f88f48-b1f8-4af9-bb6f-bd4dbadef2f1 None None] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:09:27.858546 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21fd5dee-b910-4862-bbee-afa0e2633c51 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34a98d75d84b4c67981377e0f26eca62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd70d25bf69314820b8aaecb65576978a', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:09:27.859658 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fd5dee-b910-4862-bbee-afa0e2633c51 service nova] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:09:27.860175 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fd5dee-b910-4862-bbee-afa0e2633c51 service nova] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:09:27.860619 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fd5dee-b910-4862-bbee-afa0e2633c51 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:09:27.871476 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21fd5dee-b910-4862-bbee-afa0e2633c51 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:09:27.873201 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21fd5dee-b910-4862-bbee-afa0e2633c51 service nova] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:09:27.885284 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 684/1379] 10.222.0.23 () {60 vars in 1365 bytes} [Thu May 19 12:09:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:09:30.504986 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00033c4c-4581-43ac-a3e2-dd36a1aae59c None None] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:09:30.505580 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00033c4c-4581-43ac-a3e2-dd36a1aae59c None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:09:30.505848 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00033c4c-4581-43ac-a3e2-dd36a1aae59c None None] PATH_INFO: `/` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:09:30.507548 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 696/1380] 10.222.0.23 () {54 vars in 926 bytes} [Thu May 19 12:09:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 12:09:30.513230 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d8ef0d-5e33-4c4b-9b2c-790aa1bb8657 None None] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:09:30.513699 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d8ef0d-5e33-4c4b-9b2c-790aa1bb8657 None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:09:30.513807 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d8ef0d-5e33-4c4b-9b2c-790aa1bb8657 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:09:30.533613 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.auth.core [None req-87d8ef0d-5e33-4c4b-9b2c-790aa1bb8657 None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:09:30.535106 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-87d8ef0d-5e33-4c4b-9b2c-790aa1bb8657 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:09:30.543410 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 685/1381] 10.222.0.23 () {58 vars in 1023 bytes} [Thu May 19 12:09:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 12:09:30.589443 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00033c4c-4581-43ac-a3e2-dd36a1aae59c None None] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:09:30.604511 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00033c4c-4581-43ac-a3e2-dd36a1aae59c None None] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:09:30.610913 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715ff48e-1655-4126-827e-6643f03e32c7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34a98d75d84b4c67981377e0f26eca62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd70d25bf69314820b8aaecb65576978a', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:09:30.611569 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715ff48e-1655-4126-827e-6643f03e32c7 service nova] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:09:30.611938 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715ff48e-1655-4126-827e-6643f03e32c7 service nova] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:09:30.612172 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715ff48e-1655-4126-827e-6643f03e32c7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:09:30.621435 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715ff48e-1655-4126-827e-6643f03e32c7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:09:30.623052 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715ff48e-1655-4126-827e-6643f03e32c7 service nova] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:09:30.634165 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 697/1382] 10.222.0.23 () {60 vars in 1365 bytes} [Thu May 19 12:09:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:09:33.230253 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04779d68-1f5d-4038-942f-34a4eac270f9 None None] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:09:33.231182 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04779d68-1f5d-4038-942f-34a4eac270f9 None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:09:33.231463 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04779d68-1f5d-4038-942f-34a4eac270f9 None None] PATH_INFO: `/` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:09:33.232868 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 686/1383] 10.222.0.23 () {54 vars in 926 bytes} [Thu May 19 12:09:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 12:09:33.238590 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a5aa2e-513d-4870-8bb1-06f288abedcc None None] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:09:33.239126 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a5aa2e-513d-4870-8bb1-06f288abedcc None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:09:33.239237 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a5aa2e-513d-4870-8bb1-06f288abedcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:09:33.257585 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.auth.core [None req-d8a5aa2e-513d-4870-8bb1-06f288abedcc None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:09:33.258791 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-d8a5aa2e-513d-4870-8bb1-06f288abedcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:09:33.268193 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 698/1384] 10.222.0.23 () {58 vars in 1023 bytes} [Thu May 19 12:09:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:09:33.309726 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04779d68-1f5d-4038-942f-34a4eac270f9 None None] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:09:33.325673 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04779d68-1f5d-4038-942f-34a4eac270f9 None None] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:09:33.332039 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44a0b61-21bb-4f9e-ab61-5fc2c0f13441 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34a98d75d84b4c67981377e0f26eca62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd70d25bf69314820b8aaecb65576978a', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:09:33.332829 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44a0b61-21bb-4f9e-ab61-5fc2c0f13441 service nova] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:09:33.333214 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44a0b61-21bb-4f9e-ab61-5fc2c0f13441 service nova] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:09:33.333595 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44a0b61-21bb-4f9e-ab61-5fc2c0f13441 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:09:33.348262 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44a0b61-21bb-4f9e-ab61-5fc2c0f13441 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:09:33.350138 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44a0b61-21bb-4f9e-ab61-5fc2c0f13441 service nova] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:09:33.361600 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 687/1385] 10.222.0.23 () {60 vars in 1365 bytes} [Thu May 19 12:09:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:09:35.684178 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781cc488-fc1d-44cd-9fd5-11b0e876838a None None] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:09:35.684759 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781cc488-fc1d-44cd-9fd5-11b0e876838a None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:09:35.684759 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781cc488-fc1d-44cd-9fd5-11b0e876838a None None] PATH_INFO: `/` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:09:35.686119 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 699/1386] 10.222.0.23 () {54 vars in 926 bytes} [Thu May 19 12:09:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 12:09:35.692622 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768c9a4e-d718-465b-84bc-c512d3be6eef None None] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:09:35.693434 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768c9a4e-d718-465b-84bc-c512d3be6eef None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:09:35.693714 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768c9a4e-d718-465b-84bc-c512d3be6eef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:09:35.712471 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.auth.core [None req-768c9a4e-d718-465b-84bc-c512d3be6eef None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:09:35.713689 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-768c9a4e-d718-465b-84bc-c512d3be6eef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:09:35.723365 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 688/1387] 10.222.0.23 () {58 vars in 1023 bytes} [Thu May 19 12:09:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:09:35.773003 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781cc488-fc1d-44cd-9fd5-11b0e876838a None None] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:09:35.790961 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781cc488-fc1d-44cd-9fd5-11b0e876838a None None] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:09:35.798393 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e966123d-8d47-4225-9349-b4c558f83695 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34a98d75d84b4c67981377e0f26eca62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd70d25bf69314820b8aaecb65576978a', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:09:35.799050 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e966123d-8d47-4225-9349-b4c558f83695 service nova] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:09:35.799675 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e966123d-8d47-4225-9349-b4c558f83695 service nova] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:09:35.799675 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e966123d-8d47-4225-9349-b4c558f83695 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:09:35.808547 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e966123d-8d47-4225-9349-b4c558f83695 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:09:35.810144 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e966123d-8d47-4225-9349-b4c558f83695 service nova] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:09:35.820670 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 700/1388] 10.222.0.23 () {60 vars in 1365 bytes} [Thu May 19 12:09:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 12:09:38.600256 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5376591-1cc8-43d7-a9b8-2076e77363d6 None None] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:09:38.600788 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5376591-1cc8-43d7-a9b8-2076e77363d6 None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:09:38.600888 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5376591-1cc8-43d7-a9b8-2076e77363d6 None None] PATH_INFO: `/` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:09:38.602260 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 689/1389] 10.222.0.23 () {54 vars in 926 bytes} [Thu May 19 12:09:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 12:09:38.607966 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd184f1f-4cd1-4b64-b8e0-cf1b6ea70687 None None] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:09:38.609044 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd184f1f-4cd1-4b64-b8e0-cf1b6ea70687 None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:09:38.609763 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd184f1f-4cd1-4b64-b8e0-cf1b6ea70687 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:09:38.629414 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.auth.core [None req-dd184f1f-4cd1-4b64-b8e0-cf1b6ea70687 None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:09:38.630701 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-dd184f1f-4cd1-4b64-b8e0-cf1b6ea70687 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:09:38.644717 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 701/1390] 10.222.0.23 () {58 vars in 1023 bytes} [Thu May 19 12:09:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:09:38.691723 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5376591-1cc8-43d7-a9b8-2076e77363d6 None None] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:09:38.707942 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5376591-1cc8-43d7-a9b8-2076e77363d6 None None] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:09:38.713923 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6a767b-ad3f-462c-8254-ab609014942b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34a98d75d84b4c67981377e0f26eca62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd70d25bf69314820b8aaecb65576978a', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:09:38.714587 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6a767b-ad3f-462c-8254-ab609014942b service nova] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:09:38.714887 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6a767b-ad3f-462c-8254-ab609014942b service nova] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:09:38.715174 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6a767b-ad3f-462c-8254-ab609014942b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:09:38.725067 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6a767b-ad3f-462c-8254-ab609014942b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:09:38.726712 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6a767b-ad3f-462c-8254-ab609014942b service nova] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:09:38.737830 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 690/1391] 10.222.0.23 () {60 vars in 1365 bytes} [Thu May 19 12:09:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 12:09:41.557001 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8549d711-2dea-4db8-bd8f-b680642ede24 None None] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:09:41.558036 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8549d711-2dea-4db8-bd8f-b680642ede24 None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:09:41.558036 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8549d711-2dea-4db8-bd8f-b680642ede24 None None] PATH_INFO: `/` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:09:41.559315 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 702/1392] 10.222.0.23 () {54 vars in 926 bytes} [Thu May 19 12:09:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 12:09:41.567057 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67521bfd-29b1-4340-ab15-78e80ad9e87d None None] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:09:41.567947 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67521bfd-29b1-4340-ab15-78e80ad9e87d None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:09:41.568073 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67521bfd-29b1-4340-ab15-78e80ad9e87d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:09:41.579314 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=87 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:09:41.594827 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.auth.core [None req-67521bfd-29b1-4340-ab15-78e80ad9e87d None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:09:41.607655 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-67521bfd-29b1-4340-ab15-78e80ad9e87d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:09:41.607655 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 691/1393] 10.222.0.23 () {58 vars in 1023 bytes} [Thu May 19 12:09:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:09:41.652652 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8549d711-2dea-4db8-bd8f-b680642ede24 None None] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:09:41.670584 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8549d711-2dea-4db8-bd8f-b680642ede24 None None] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:09:41.676885 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f3bbe5-d5c4-48fa-9094-903b0b877040 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34a98d75d84b4c67981377e0f26eca62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd70d25bf69314820b8aaecb65576978a', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:09:41.677540 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f3bbe5-d5c4-48fa-9094-903b0b877040 service nova] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:09:41.677843 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f3bbe5-d5c4-48fa-9094-903b0b877040 service nova] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:09:41.678121 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f3bbe5-d5c4-48fa-9094-903b0b877040 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:09:41.688560 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f3bbe5-d5c4-48fa-9094-903b0b877040 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:09:41.690183 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f3bbe5-d5c4-48fa-9094-903b0b877040 service nova] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:09:41.701504 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 703/1394] 10.222.0.23 () {60 vars in 1365 bytes} [Thu May 19 12:09:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 12:09:42.475336 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6193f5-76e0-4bf6-8227-0e0f713e9083 None None] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:09:42.476034 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6193f5-76e0-4bf6-8227-0e0f713e9083 None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:09:42.476631 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6193f5-76e0-4bf6-8227-0e0f713e9083 None None] PATH_INFO: `/` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:09:42.479511 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 692/1395] 10.222.0.23 () {54 vars in 926 bytes} [Thu May 19 12:09:42 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 12:09:42.486710 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c30500-5160-427b-b258-c57b9ef24734 None None] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:09:42.487039 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c30500-5160-427b-b258-c57b9ef24734 None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:09:42.487354 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c30500-5160-427b-b258-c57b9ef24734 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:09:42.508456 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.auth.core [None req-e4c30500-5160-427b-b258-c57b9ef24734 None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:09:42.509719 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-e4c30500-5160-427b-b258-c57b9ef24734 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:09:42.519517 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 704/1396] 10.222.0.23 () {58 vars in 1023 bytes} [Thu May 19 12:09:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 12:09:42.583258 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8077f4-c6f5-4989-a30c-b788c85b1445 None None] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:09:42.583933 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8077f4-c6f5-4989-a30c-b788c85b1445 None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:09:42.583933 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8077f4-c6f5-4989-a30c-b788c85b1445 None None] PATH_INFO: `/` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:09:42.585299 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 693/1397] 10.222.0.23 () {54 vars in 962 bytes} [Thu May 19 12:09:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 12:09:42.590978 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d83559-e334-4d9f-9258-90987e1e5dd2 None None] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:09:42.591530 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d83559-e334-4d9f-9258-90987e1e5dd2 None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:09:42.591530 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d83559-e334-4d9f-9258-90987e1e5dd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:09:42.651120 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.auth.core [None req-b0d83559-e334-4d9f-9258-90987e1e5dd2 None None] MFA Rules not processed for user `21697802f2be42f9baa8e07490d5ebdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:09:42.652368 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-b0d83559-e334-4d9f-9258-90987e1e5dd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:09:42.729346 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 705/1398] 10.222.0.23 () {58 vars in 1059 bytes} [Thu May 19 12:09:42 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:09:42.738441 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8077f4-c6f5-4989-a30c-b788c85b1445 None None] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:09:42.749794 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8077f4-c6f5-4989-a30c-b788c85b1445 None None] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:09:42.753373 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b80207-069a-402b-8602-0c7658e5294a service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '21697802f2be42f9baa8e07490d5ebdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd70d25bf69314820b8aaecb65576978a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:09:42.754309 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b80207-069a-402b-8602-0c7658e5294a service glance] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:09:42.754595 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b80207-069a-402b-8602-0c7658e5294a service glance] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:09:42.754871 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b80207-069a-402b-8602-0c7658e5294a service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:09:42.766365 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b80207-069a-402b-8602-0c7658e5294a service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:09:42.768001 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b80207-069a-402b-8602-0c7658e5294a service glance] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:09:42.778918 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 694/1399] 10.222.0.23 () {60 vars in 1365 bytes} [Thu May 19 12:09:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:09:43.514387 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44c11c1-c839-4c8a-90c8-0120982e7c43 None None] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:09:43.515366 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44c11c1-c839-4c8a-90c8-0120982e7c43 None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:09:43.515708 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44c11c1-c839-4c8a-90c8-0120982e7c43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:09:43.540607 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:09:43.588838 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.auth.core [None req-c44c11c1-c839-4c8a-90c8-0120982e7c43 None None] MFA Rules not processed for user `0c3181fb4f584a549a786adfefdf3dfb`. Rule list: `[]` (Enabled: `True`). {{(pid=80540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:09:43.590643 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-c44c11c1-c839-4c8a-90c8-0120982e7c43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:09:43.677442 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 706/1400] 10.222.0.23 () {58 vars in 1009 bytes} [Thu May 19 12:09:43 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:09:43.726107 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38dcab2d-803b-4e53-a2eb-dcce9f0ac3db None None] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:09:43.726817 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38dcab2d-803b-4e53-a2eb-dcce9f0ac3db None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:09:43.726935 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38dcab2d-803b-4e53-a2eb-dcce9f0ac3db None None] PATH_INFO: `/` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:09:43.728354 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 695/1401] 10.222.0.23 () {54 vars in 942 bytes} [Thu May 19 12:09:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 12:09:43.736111 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be6c8d5-b546-4911-a11f-097228c97b84 None None] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:09:43.736485 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be6c8d5-b546-4911-a11f-097228c97b84 None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:09:43.736837 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be6c8d5-b546-4911-a11f-097228c97b84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:09:43.798191 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.auth.core [None req-3be6c8d5-b546-4911-a11f-097228c97b84 None None] MFA Rules not processed for user `15cbfc73ae1243d2a8b1e70f9cd99876`. Rule list: `[]` (Enabled: `True`). {{(pid=80540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:09:43.799334 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-3be6c8d5-b546-4911-a11f-097228c97b84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:09:43.885925 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 707/1402] 10.222.0.23 () {58 vars in 1039 bytes} [Thu May 19 12:09:43 2022] POST /identity/v3/auth/tokens => generated 3294 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:09:43.895428 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38dcab2d-803b-4e53-a2eb-dcce9f0ac3db None None] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:09:43.909961 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38dcab2d-803b-4e53-a2eb-dcce9f0ac3db None None] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:09:43.915280 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22aba0de-8f50-4fe7-80f3-67c703066bac service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15cbfc73ae1243d2a8b1e70f9cd99876', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd70d25bf69314820b8aaecb65576978a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:09:43.916016 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22aba0de-8f50-4fe7-80f3-67c703066bac service swift] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:09:43.916511 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22aba0de-8f50-4fe7-80f3-67c703066bac service swift] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:09:43.916798 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22aba0de-8f50-4fe7-80f3-67c703066bac service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:09:43.927134 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22aba0de-8f50-4fe7-80f3-67c703066bac service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:09:43.928914 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22aba0de-8f50-4fe7-80f3-67c703066bac service swift] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:09:43.939108 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 696/1403] 10.222.0.23 () {60 vars in 1384 bytes} [Thu May 19 12:09:43 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 19 12:09:44.647998 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432d4501-4814-42b3-bb25-592c515394a2 None None] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:09:44.648414 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432d4501-4814-42b3-bb25-592c515394a2 None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:09:44.648612 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432d4501-4814-42b3-bb25-592c515394a2 None None] PATH_INFO: `/` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:09:44.649954 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 708/1404] 10.222.0.23 () {54 vars in 926 bytes} [Thu May 19 12:09:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 12:09:44.656375 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea2f0c9-3a39-4c6b-a06d-9018e643de74 None None] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:09:44.656649 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea2f0c9-3a39-4c6b-a06d-9018e643de74 None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:09:44.657010 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea2f0c9-3a39-4c6b-a06d-9018e643de74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:09:44.675250 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.auth.core [None req-1ea2f0c9-3a39-4c6b-a06d-9018e643de74 None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:09:44.676483 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-1ea2f0c9-3a39-4c6b-a06d-9018e643de74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:09:44.688622 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 697/1405] 10.222.0.23 () {58 vars in 1023 bytes} [Thu May 19 12:09:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:09:44.736632 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432d4501-4814-42b3-bb25-592c515394a2 None None] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:09:44.754662 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432d4501-4814-42b3-bb25-592c515394a2 None None] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:09:44.760718 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e584866e-978a-4ed3-9ed1-6bc411a850ed service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34a98d75d84b4c67981377e0f26eca62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd70d25bf69314820b8aaecb65576978a', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:09:44.761448 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e584866e-978a-4ed3-9ed1-6bc411a850ed service nova] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:09:44.761738 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e584866e-978a-4ed3-9ed1-6bc411a850ed service nova] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:09:44.762016 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e584866e-978a-4ed3-9ed1-6bc411a850ed service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:09:44.771966 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e584866e-978a-4ed3-9ed1-6bc411a850ed service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:09:44.776311 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e584866e-978a-4ed3-9ed1-6bc411a850ed service nova] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:09:44.787524 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 709/1406] 10.222.0.23 () {60 vars in 1365 bytes} [Thu May 19 12:09:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:09:47.464224 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d52899-c69e-4c70-bd80-be1a2223e04b None None] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:09:47.464809 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d52899-c69e-4c70-bd80-be1a2223e04b None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:09:47.464809 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d52899-c69e-4c70-bd80-be1a2223e04b None None] PATH_INFO: `/` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:09:47.466227 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 698/1407] 10.222.0.23 () {54 vars in 926 bytes} [Thu May 19 12:09:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 12:09:47.472672 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c06c3e5-e492-41d7-bc39-64a7979aa3da None None] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:09:47.473166 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c06c3e5-e492-41d7-bc39-64a7979aa3da None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:09:47.473283 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c06c3e5-e492-41d7-bc39-64a7979aa3da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:09:47.493594 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.auth.core [None req-9c06c3e5-e492-41d7-bc39-64a7979aa3da None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:09:47.494883 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-9c06c3e5-e492-41d7-bc39-64a7979aa3da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:09:47.507302 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 710/1408] 10.222.0.23 () {58 vars in 1023 bytes} [Thu May 19 12:09:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:09:47.556788 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d52899-c69e-4c70-bd80-be1a2223e04b None None] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:09:47.577819 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d52899-c69e-4c70-bd80-be1a2223e04b None None] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:09:47.587110 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1b436d-8228-43e5-9af9-37eafbf47fd7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34a98d75d84b4c67981377e0f26eca62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd70d25bf69314820b8aaecb65576978a', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:09:47.587775 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1b436d-8228-43e5-9af9-37eafbf47fd7 service nova] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:09:47.588091 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1b436d-8228-43e5-9af9-37eafbf47fd7 service nova] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:09:47.588532 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1b436d-8228-43e5-9af9-37eafbf47fd7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:09:47.598513 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1b436d-8228-43e5-9af9-37eafbf47fd7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:09:47.600171 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1b436d-8228-43e5-9af9-37eafbf47fd7 service nova] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:09:47.611768 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 699/1409] 10.222.0.23 () {60 vars in 1365 bytes} [Thu May 19 12:09:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:09:57.488145 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=87 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:09:57.606644 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=23 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:10:10.068541 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f87636a-6988-4c4b-b287-c4b733eef2c0 None None] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:10:10.069188 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f87636a-6988-4c4b-b287-c4b733eef2c0 None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:10:10.069476 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f87636a-6988-4c4b-b287-c4b733eef2c0 None None] PATH_INFO: `/` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:10:10.070937 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 711/1410] 10.222.0.23 () {54 vars in 918 bytes} [Thu May 19 12:10:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 12:10:10.076264 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2701de63-9f40-4af5-b56c-66e404464b33 None None] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:10:10.076892 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2701de63-9f40-4af5-b56c-66e404464b33 None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:10:10.076892 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2701de63-9f40-4af5-b56c-66e404464b33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:10:10.096558 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.auth.core [None req-2701de63-9f40-4af5-b56c-66e404464b33 None None] MFA Rules not processed for user `50f937eed4f3414fa6f19cab2a84eb5f`. Rule list: `[]` (Enabled: `True`). {{(pid=80539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:10:10.097770 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-2701de63-9f40-4af5-b56c-66e404464b33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:10:10.106408 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 700/1411] 10.222.0.23 () {58 vars in 1015 bytes} [Thu May 19 12:10:10 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:10:10.122841 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e873cf1-1334-4416-95e6-bfb715bf45ae None None] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:10:10.123131 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e873cf1-1334-4416-95e6-bfb715bf45ae None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:10:10.123447 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e873cf1-1334-4416-95e6-bfb715bf45ae None None] PATH_INFO: `/` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:10:10.126089 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 712/1412] 10.222.0.23 () {54 vars in 918 bytes} [Thu May 19 12:10:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 12:10:10.131251 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3374246a-e34f-4c50-8d4e-6c98570c7dc9 None None] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:10:10.131542 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3374246a-e34f-4c50-8d4e-6c98570c7dc9 None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:10:10.131839 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3374246a-e34f-4c50-8d4e-6c98570c7dc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:10:10.150338 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.auth.core [None req-3374246a-e34f-4c50-8d4e-6c98570c7dc9 None None] MFA Rules not processed for user `34a98d75d84b4c67981377e0f26eca62`. Rule list: `[]` (Enabled: `True`). {{(pid=80539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:10:10.151763 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-3374246a-e34f-4c50-8d4e-6c98570c7dc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:10:10.161913 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 701/1413] 10.222.0.23 () {58 vars in 1015 bytes} [Thu May 19 12:10:10 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:10:12.951088 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbf2998-4f2a-4f1c-b19d-cc6f286f0015 None None] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:10:12.951514 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbf2998-4f2a-4f1c-b19d-cc6f286f0015 None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:10:12.951721 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbf2998-4f2a-4f1c-b19d-cc6f286f0015 None None] PATH_INFO: `/` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:10:12.953334 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 713/1414] 127.0.0.1 () {54 vars in 916 bytes} [Thu May 19 12:10:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 12:10:12.959392 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d528e56d-ebfb-4bd7-b004-211a8bc209b7 None None] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:10:12.959877 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d528e56d-ebfb-4bd7-b004-211a8bc209b7 None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:10:12.960108 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d528e56d-ebfb-4bd7-b004-211a8bc209b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:10:12.979069 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.auth.core [None req-d528e56d-ebfb-4bd7-b004-211a8bc209b7 None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:10:12.980312 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-d528e56d-ebfb-4bd7-b004-211a8bc209b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:10:12.988914 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 702/1415] 127.0.0.1 () {58 vars in 1013 bytes} [Thu May 19 12:10:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:10:13.035047 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbf2998-4f2a-4f1c-b19d-cc6f286f0015 None None] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:10:13.050565 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbf2998-4f2a-4f1c-b19d-cc6f286f0015 None None] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:10:13.059515 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408333b3-eac4-4289-81b0-392778541493 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34a98d75d84b4c67981377e0f26eca62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd70d25bf69314820b8aaecb65576978a', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:10:13.060197 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408333b3-eac4-4289-81b0-392778541493 service nova] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:10:13.060480 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408333b3-eac4-4289-81b0-392778541493 service nova] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:10:13.060751 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408333b3-eac4-4289-81b0-392778541493 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:10:13.071044 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408333b3-eac4-4289-81b0-392778541493 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:10:13.072753 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408333b3-eac4-4289-81b0-392778541493 service nova] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:10:13.086670 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 714/1416] 10.222.0.23 () {60 vars in 1365 bytes} [Thu May 19 12:10:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:10:14.110318 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2056e5-8d0f-4812-93c9-f5929b88f1e8 None None] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:10:14.110852 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2056e5-8d0f-4812-93c9-f5929b88f1e8 None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:10:14.110852 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2056e5-8d0f-4812-93c9-f5929b88f1e8 None None] PATH_INFO: `/` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:10:14.112295 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 703/1417] 127.0.0.1 () {54 vars in 901 bytes} [Thu May 19 12:10:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 12:10:14.118156 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355f373e-124b-41ca-9ea9-fbfd4e3e574a None None] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:10:14.118641 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355f373e-124b-41ca-9ea9-fbfd4e3e574a None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:10:14.118779 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355f373e-124b-41ca-9ea9-fbfd4e3e574a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:10:14.138276 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.auth.core [None req-355f373e-124b-41ca-9ea9-fbfd4e3e574a None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:10:14.140175 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-355f373e-124b-41ca-9ea9-fbfd4e3e574a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:10:14.148701 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 715/1418] 127.0.0.1 () {58 vars in 998 bytes} [Thu May 19 12:10:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:10:14.313899 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2056e5-8d0f-4812-93c9-f5929b88f1e8 None None] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:10:14.328977 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2056e5-8d0f-4812-93c9-f5929b88f1e8 None None] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:10:14.335027 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b7673f-701d-475c-9128-f45a468ae227 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34a98d75d84b4c67981377e0f26eca62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd70d25bf69314820b8aaecb65576978a', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:10:14.335679 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b7673f-701d-475c-9128-f45a468ae227 service nova] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:10:14.335993 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b7673f-701d-475c-9128-f45a468ae227 service nova] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:10:14.336295 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b7673f-701d-475c-9128-f45a468ae227 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:10:14.345996 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b7673f-701d-475c-9128-f45a468ae227 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:10:14.347726 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b7673f-701d-475c-9128-f45a468ae227 service nova] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:10:14.359180 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 704/1419] 10.222.0.23 () {60 vars in 1365 bytes} [Thu May 19 12:10:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:10:15.332671 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827f8231-ea61-46c1-80e4-6ced63d55c4d None None] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:10:15.333226 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827f8231-ea61-46c1-80e4-6ced63d55c4d None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:10:15.333226 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827f8231-ea61-46c1-80e4-6ced63d55c4d None None] PATH_INFO: `/` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:10:15.334613 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 716/1420] 127.0.0.1 () {54 vars in 901 bytes} [Thu May 19 12:10:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 12:10:15.340100 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779d2405-eb87-446b-b7aa-94932778dd11 None None] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:10:15.340583 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779d2405-eb87-446b-b7aa-94932778dd11 None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:10:15.340690 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779d2405-eb87-446b-b7aa-94932778dd11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:10:15.360191 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.auth.core [None req-779d2405-eb87-446b-b7aa-94932778dd11 None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:10:15.361437 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-779d2405-eb87-446b-b7aa-94932778dd11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:10:15.369285 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 705/1421] 127.0.0.1 () {58 vars in 998 bytes} [Thu May 19 12:10:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:10:15.521520 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827f8231-ea61-46c1-80e4-6ced63d55c4d None None] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:10:15.536277 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827f8231-ea61-46c1-80e4-6ced63d55c4d None None] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:10:15.541777 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd0d826-3108-4648-b53a-7e9d81686362 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34a98d75d84b4c67981377e0f26eca62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd70d25bf69314820b8aaecb65576978a', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:10:15.546243 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd0d826-3108-4648-b53a-7e9d81686362 service nova] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:10:15.546243 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd0d826-3108-4648-b53a-7e9d81686362 service nova] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:10:15.546243 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd0d826-3108-4648-b53a-7e9d81686362 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:10:15.552211 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd0d826-3108-4648-b53a-7e9d81686362 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:10:15.553826 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd0d826-3108-4648-b53a-7e9d81686362 service nova] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:10:15.564367 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 717/1422] 10.222.0.23 () {60 vars in 1365 bytes} [Thu May 19 12:10:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:10:16.533085 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f9be36-6255-4009-a842-bf3b33836ea9 None None] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:10:16.533637 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f9be36-6255-4009-a842-bf3b33836ea9 None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:10:16.533637 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f9be36-6255-4009-a842-bf3b33836ea9 None None] PATH_INFO: `/` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:10:16.535057 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 706/1423] 127.0.0.1 () {54 vars in 901 bytes} [Thu May 19 12:10:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 12:10:16.540243 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c343fdfe-03ee-4197-9dcd-4c473f8c5877 None None] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:10:16.540709 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c343fdfe-03ee-4197-9dcd-4c473f8c5877 None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:10:16.547507 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c343fdfe-03ee-4197-9dcd-4c473f8c5877 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:10:16.559797 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.auth.core [None req-c343fdfe-03ee-4197-9dcd-4c473f8c5877 None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:10:16.561091 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-c343fdfe-03ee-4197-9dcd-4c473f8c5877 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:10:16.569304 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 718/1424] 127.0.0.1 () {58 vars in 998 bytes} [Thu May 19 12:10:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:10:16.724480 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f9be36-6255-4009-a842-bf3b33836ea9 None None] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:10:16.739051 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f9be36-6255-4009-a842-bf3b33836ea9 None None] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:10:16.744508 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78992a2-c433-4458-ad3a-1d82ed6451ab service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34a98d75d84b4c67981377e0f26eca62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd70d25bf69314820b8aaecb65576978a', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:10:16.745152 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78992a2-c433-4458-ad3a-1d82ed6451ab service nova] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:10:16.745439 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78992a2-c433-4458-ad3a-1d82ed6451ab service nova] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:10:16.745718 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78992a2-c433-4458-ad3a-1d82ed6451ab service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:10:16.754417 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78992a2-c433-4458-ad3a-1d82ed6451ab service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:10:16.756042 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78992a2-c433-4458-ad3a-1d82ed6451ab service nova] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:10:16.766876 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 707/1425] 10.222.0.23 () {60 vars in 1365 bytes} [Thu May 19 12:10:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:10:17.735092 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afd65f4-42c2-4eb5-89b4-1df6a4f73607 None None] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:10:17.735640 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afd65f4-42c2-4eb5-89b4-1df6a4f73607 None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:10:17.735640 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afd65f4-42c2-4eb5-89b4-1df6a4f73607 None None] PATH_INFO: `/` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:10:17.736990 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 719/1426] 127.0.0.1 () {54 vars in 901 bytes} [Thu May 19 12:10:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 12:10:17.742054 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631b8bed-fb01-45da-8c39-9f5805f9b7ac None None] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:10:17.742516 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631b8bed-fb01-45da-8c39-9f5805f9b7ac None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:10:17.742608 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631b8bed-fb01-45da-8c39-9f5805f9b7ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:10:17.762750 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.auth.core [None req-631b8bed-fb01-45da-8c39-9f5805f9b7ac None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:10:17.764662 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-631b8bed-fb01-45da-8c39-9f5805f9b7ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:10:17.772868 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 708/1427] 127.0.0.1 () {58 vars in 998 bytes} [Thu May 19 12:10:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:10:17.927401 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9afd65f4-42c2-4eb5-89b4-1df6a4f73607 None None] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:10:17.942943 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9afd65f4-42c2-4eb5-89b4-1df6a4f73607 None None] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:10:17.949329 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3fabf3-1585-407f-ae25-0f9166716b8d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34a98d75d84b4c67981377e0f26eca62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd70d25bf69314820b8aaecb65576978a', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:10:17.949979 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3fabf3-1585-407f-ae25-0f9166716b8d service nova] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:10:17.950261 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3fabf3-1585-407f-ae25-0f9166716b8d service nova] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:10:17.950537 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3fabf3-1585-407f-ae25-0f9166716b8d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:10:17.961214 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3fabf3-1585-407f-ae25-0f9166716b8d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:10:17.963078 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3fabf3-1585-407f-ae25-0f9166716b8d service nova] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:10:17.974964 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 720/1428] 10.222.0.23 () {60 vars in 1365 bytes} [Thu May 19 12:10:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:10:18.939591 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4531c473-f05e-4713-95f7-6a319be7e5e5 None None] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:10:18.940196 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4531c473-f05e-4713-95f7-6a319be7e5e5 None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:10:18.940295 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4531c473-f05e-4713-95f7-6a319be7e5e5 None None] PATH_INFO: `/` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:10:18.941655 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 709/1429] 127.0.0.1 () {54 vars in 901 bytes} [Thu May 19 12:10:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 12:10:18.947527 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6410fd77-98bf-4b2c-a09d-d045d7a741dc None None] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:10:18.948168 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6410fd77-98bf-4b2c-a09d-d045d7a741dc None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:10:18.948460 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6410fd77-98bf-4b2c-a09d-d045d7a741dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:10:18.969642 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.auth.core [None req-6410fd77-98bf-4b2c-a09d-d045d7a741dc None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:10:18.970857 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-6410fd77-98bf-4b2c-a09d-d045d7a741dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:10:18.978897 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 721/1430] 127.0.0.1 () {58 vars in 998 bytes} [Thu May 19 12:10:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 12:10:19.134704 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4531c473-f05e-4713-95f7-6a319be7e5e5 None None] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:10:19.149066 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4531c473-f05e-4713-95f7-6a319be7e5e5 None None] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:10:19.154414 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3772120c-7180-409c-b567-d5de1554ccfc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34a98d75d84b4c67981377e0f26eca62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd70d25bf69314820b8aaecb65576978a', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:10:19.155617 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3772120c-7180-409c-b567-d5de1554ccfc service nova] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:10:19.155943 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3772120c-7180-409c-b567-d5de1554ccfc service nova] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:10:19.156269 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3772120c-7180-409c-b567-d5de1554ccfc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:10:19.165607 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3772120c-7180-409c-b567-d5de1554ccfc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:10:19.167223 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3772120c-7180-409c-b567-d5de1554ccfc service nova] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:10:19.178372 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 710/1431] 10.222.0.23 () {60 vars in 1365 bytes} [Thu May 19 12:10:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:10:20.150413 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a7956f-fd3c-46c6-b19c-93ed78f53373 None None] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:10:20.150980 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a7956f-fd3c-46c6-b19c-93ed78f53373 None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:10:20.150980 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a7956f-fd3c-46c6-b19c-93ed78f53373 None None] PATH_INFO: `/` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:10:20.152379 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 722/1432] 127.0.0.1 () {54 vars in 901 bytes} [Thu May 19 12:10:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 12:10:20.157678 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565e957d-0744-497b-b885-1256a60e10eb None None] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:10:20.158135 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565e957d-0744-497b-b885-1256a60e10eb None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:10:20.158245 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565e957d-0744-497b-b885-1256a60e10eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:10:20.176575 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.auth.core [None req-565e957d-0744-497b-b885-1256a60e10eb None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:10:20.177791 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-565e957d-0744-497b-b885-1256a60e10eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:10:20.185671 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 711/1433] 127.0.0.1 () {58 vars in 998 bytes} [Thu May 19 12:10:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:10:20.338110 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a7956f-fd3c-46c6-b19c-93ed78f53373 None None] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:10:20.352545 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a7956f-fd3c-46c6-b19c-93ed78f53373 None None] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:10:20.360720 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776bbb47-bb70-4810-9d9e-7c2c5231fe8c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34a98d75d84b4c67981377e0f26eca62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd70d25bf69314820b8aaecb65576978a', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:10:20.361363 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776bbb47-bb70-4810-9d9e-7c2c5231fe8c service nova] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:10:20.361648 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776bbb47-bb70-4810-9d9e-7c2c5231fe8c service nova] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:10:20.361922 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776bbb47-bb70-4810-9d9e-7c2c5231fe8c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:10:20.370679 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776bbb47-bb70-4810-9d9e-7c2c5231fe8c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:10:20.372367 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776bbb47-bb70-4810-9d9e-7c2c5231fe8c service nova] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:10:20.382590 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 723/1434] 10.222.0.23 () {60 vars in 1365 bytes} [Thu May 19 12:10:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:10:21.364284 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba2b127-c800-4a60-b74b-37e77f14503f None None] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:10:21.364888 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba2b127-c800-4a60-b74b-37e77f14503f None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:10:21.364888 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba2b127-c800-4a60-b74b-37e77f14503f None None] PATH_INFO: `/` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:10:21.366267 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 712/1435] 127.0.0.1 () {54 vars in 901 bytes} [Thu May 19 12:10:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 12:10:21.371280 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e9b773-5a61-4f82-aff0-b532041d177d None None] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:10:21.371678 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e9b773-5a61-4f82-aff0-b532041d177d None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:10:21.371859 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e9b773-5a61-4f82-aff0-b532041d177d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:10:21.390303 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.auth.core [None req-f9e9b773-5a61-4f82-aff0-b532041d177d None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:10:21.391492 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-f9e9b773-5a61-4f82-aff0-b532041d177d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:10:21.399284 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 724/1436] 127.0.0.1 () {58 vars in 998 bytes} [Thu May 19 12:10:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:10:21.550216 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba2b127-c800-4a60-b74b-37e77f14503f None None] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:10:21.565501 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba2b127-c800-4a60-b74b-37e77f14503f None None] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:10:21.570994 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6762de-1d22-4bc6-b10b-df39202ad614 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34a98d75d84b4c67981377e0f26eca62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd70d25bf69314820b8aaecb65576978a', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:10:21.571656 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6762de-1d22-4bc6-b10b-df39202ad614 service nova] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:10:21.571973 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6762de-1d22-4bc6-b10b-df39202ad614 service nova] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:10:21.572252 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6762de-1d22-4bc6-b10b-df39202ad614 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:10:21.581957 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6762de-1d22-4bc6-b10b-df39202ad614 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:10:21.583548 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6762de-1d22-4bc6-b10b-df39202ad614 service nova] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:10:21.593868 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 713/1437] 10.222.0.23 () {60 vars in 1365 bytes} [Thu May 19 12:10:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:10:22.557409 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6953151-1034-4901-a472-fb39ac5cb3bc None None] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:10:22.557989 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6953151-1034-4901-a472-fb39ac5cb3bc None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:10:22.557989 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6953151-1034-4901-a472-fb39ac5cb3bc None None] PATH_INFO: `/` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:10:22.559353 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 725/1438] 127.0.0.1 () {54 vars in 901 bytes} [Thu May 19 12:10:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 12:10:22.565322 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87619f9-b622-45fa-ad92-ce96819c30b3 None None] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:10:22.565912 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87619f9-b622-45fa-ad92-ce96819c30b3 None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:10:22.566019 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87619f9-b622-45fa-ad92-ce96819c30b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:10:22.584321 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.auth.core [None req-e87619f9-b622-45fa-ad92-ce96819c30b3 None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:10:22.585525 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-e87619f9-b622-45fa-ad92-ce96819c30b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:10:22.593815 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 714/1439] 127.0.0.1 () {58 vars in 998 bytes} [Thu May 19 12:10:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:10:22.750841 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6953151-1034-4901-a472-fb39ac5cb3bc None None] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:10:22.765826 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6953151-1034-4901-a472-fb39ac5cb3bc None None] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:10:22.771494 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c8a301-a9b8-4280-b060-193193cb0454 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34a98d75d84b4c67981377e0f26eca62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd70d25bf69314820b8aaecb65576978a', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:10:22.772177 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c8a301-a9b8-4280-b060-193193cb0454 service nova] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:10:22.772468 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c8a301-a9b8-4280-b060-193193cb0454 service nova] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:10:22.772750 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c8a301-a9b8-4280-b060-193193cb0454 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:10:22.781720 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c8a301-a9b8-4280-b060-193193cb0454 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:10:22.783333 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c8a301-a9b8-4280-b060-193193cb0454 service nova] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:10:22.794260 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 726/1440] 10.222.0.23 () {60 vars in 1365 bytes} [Thu May 19 12:10:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:10:24.949343 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b77f6a1-c495-4ba3-8893-addc085bea72 None None] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:10:24.950219 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b77f6a1-c495-4ba3-8893-addc085bea72 None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:10:24.951195 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b77f6a1-c495-4ba3-8893-addc085bea72 None None] PATH_INFO: `/` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:10:24.952510 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 715/1441] 127.0.0.1 () {54 vars in 916 bytes} [Thu May 19 12:10:24 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 12:10:24.957992 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60574057-73e1-42b5-b97d-b15b9e1968a6 None None] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:10:24.958469 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60574057-73e1-42b5-b97d-b15b9e1968a6 None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:10:24.958567 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60574057-73e1-42b5-b97d-b15b9e1968a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:10:24.976943 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.auth.core [None req-60574057-73e1-42b5-b97d-b15b9e1968a6 None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:10:24.978147 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-60574057-73e1-42b5-b97d-b15b9e1968a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:10:24.987028 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 727/1442] 127.0.0.1 () {58 vars in 1013 bytes} [Thu May 19 12:10:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:10:25.025686 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b77f6a1-c495-4ba3-8893-addc085bea72 None None] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:10:25.036836 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b77f6a1-c495-4ba3-8893-addc085bea72 None None] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:10:25.040471 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2fc81b6-dc83-4d5f-b3f5-896a488b0a4a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a00b3fa7497456895e5e0bf105c4fc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd70d25bf69314820b8aaecb65576978a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:10:25.041117 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fc81b6-dc83-4d5f-b3f5-896a488b0a4a service neutron] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:10:25.041405 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fc81b6-dc83-4d5f-b3f5-896a488b0a4a service neutron] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:10:25.041686 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fc81b6-dc83-4d5f-b3f5-896a488b0a4a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:10:25.051671 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2fc81b6-dc83-4d5f-b3f5-896a488b0a4a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:10:25.053185 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2fc81b6-dc83-4d5f-b3f5-896a488b0a4a service neutron] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:10:25.063626 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 716/1443] 10.222.0.23 () {60 vars in 1365 bytes} [Thu May 19 12:10:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:10:27.513077 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af5f170-54d1-4fcb-8a3c-056e0851fc0d None None] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:10:27.513661 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af5f170-54d1-4fcb-8a3c-056e0851fc0d None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:10:27.513661 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af5f170-54d1-4fcb-8a3c-056e0851fc0d None None] PATH_INFO: `/` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:10:27.515824 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 728/1444] 127.0.0.1 () {54 vars in 916 bytes} [Thu May 19 12:10:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 12:10:27.521536 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdcb561-ffd6-41bc-b261-5b6f7359d5e8 None None] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:10:27.522027 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdcb561-ffd6-41bc-b261-5b6f7359d5e8 None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:10:27.522143 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdcb561-ffd6-41bc-b261-5b6f7359d5e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:10:27.540526 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.auth.core [None req-5bdcb561-ffd6-41bc-b261-5b6f7359d5e8 None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:10:27.541756 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-5bdcb561-ffd6-41bc-b261-5b6f7359d5e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:10:27.552336 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 717/1445] 127.0.0.1 () {58 vars in 1013 bytes} [Thu May 19 12:10:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:10:27.591551 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af5f170-54d1-4fcb-8a3c-056e0851fc0d None None] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:10:27.602605 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af5f170-54d1-4fcb-8a3c-056e0851fc0d None None] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:10:27.606343 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c2e349-2274-41d7-914e-f0ef7eb2365c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a00b3fa7497456895e5e0bf105c4fc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd70d25bf69314820b8aaecb65576978a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:10:27.607043 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c2e349-2274-41d7-914e-f0ef7eb2365c service neutron] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:10:27.607257 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c2e349-2274-41d7-914e-f0ef7eb2365c service neutron] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:10:27.607543 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c2e349-2274-41d7-914e-f0ef7eb2365c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:10:27.617662 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c2e349-2274-41d7-914e-f0ef7eb2365c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:10:27.619095 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c2e349-2274-41d7-914e-f0ef7eb2365c service neutron] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:10:27.629505 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 729/1446] 10.222.0.23 () {60 vars in 1365 bytes} [Thu May 19 12:10:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:10:30.379292 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9e5d45-720b-49c0-b16e-b9c5f4f36331 None None] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:10:30.379918 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9e5d45-720b-49c0-b16e-b9c5f4f36331 None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:10:30.379918 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9e5d45-720b-49c0-b16e-b9c5f4f36331 None None] PATH_INFO: `/` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:10:30.381250 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 718/1447] 127.0.0.1 () {54 vars in 916 bytes} [Thu May 19 12:10:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 12:10:30.387053 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a99d1a7-1bec-42c6-9558-a536dba16e8e None None] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:10:30.387554 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a99d1a7-1bec-42c6-9558-a536dba16e8e None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:10:30.387686 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a99d1a7-1bec-42c6-9558-a536dba16e8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:10:30.406371 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.auth.core [None req-5a99d1a7-1bec-42c6-9558-a536dba16e8e None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:10:30.407561 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-5a99d1a7-1bec-42c6-9558-a536dba16e8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:10:30.415430 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 730/1448] 127.0.0.1 () {58 vars in 1013 bytes} [Thu May 19 12:10:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:10:30.457073 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9e5d45-720b-49c0-b16e-b9c5f4f36331 None None] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:10:30.458775 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-fd9e5d45-720b-49c0-b16e-b9c5f4f36331 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:10:30.472042 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9e5d45-720b-49c0-b16e-b9c5f4f36331 None None] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:10:30.475565 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696c18c5-1e98-4116-8fa4-c8c20f103bcd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a00b3fa7497456895e5e0bf105c4fc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd70d25bf69314820b8aaecb65576978a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:10:30.476243 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696c18c5-1e98-4116-8fa4-c8c20f103bcd service neutron] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:10:30.476532 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696c18c5-1e98-4116-8fa4-c8c20f103bcd service neutron] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:10:30.476814 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696c18c5-1e98-4116-8fa4-c8c20f103bcd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:10:30.486455 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696c18c5-1e98-4116-8fa4-c8c20f103bcd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:10:30.487923 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696c18c5-1e98-4116-8fa4-c8c20f103bcd service neutron] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:10:30.497906 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 719/1449] 10.222.0.23 () {60 vars in 1365 bytes} [Thu May 19 12:10:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:10:32.926175 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f9d87e-a06e-4f5e-972c-532028f23560 None None] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:10:32.926771 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f9d87e-a06e-4f5e-972c-532028f23560 None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:10:32.926771 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f9d87e-a06e-4f5e-972c-532028f23560 None None] PATH_INFO: `/` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:10:32.928138 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 731/1450] 127.0.0.1 () {54 vars in 916 bytes} [Thu May 19 12:10:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 12:10:32.933830 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e917c191-0502-4b32-b9ed-d78f06c45d0d None None] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:10:32.934308 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e917c191-0502-4b32-b9ed-d78f06c45d0d None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:10:32.934411 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e917c191-0502-4b32-b9ed-d78f06c45d0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:10:32.954214 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.auth.core [None req-e917c191-0502-4b32-b9ed-d78f06c45d0d None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:10:32.955423 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-e917c191-0502-4b32-b9ed-d78f06c45d0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:10:32.963702 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 720/1451] 127.0.0.1 () {58 vars in 1013 bytes} [Thu May 19 12:10:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:10:33.003276 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f9d87e-a06e-4f5e-972c-532028f23560 None None] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:10:33.016322 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f9d87e-a06e-4f5e-972c-532028f23560 None None] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:10:33.019865 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac39dee-9afa-4519-becc-6ab5291c084e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a00b3fa7497456895e5e0bf105c4fc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd70d25bf69314820b8aaecb65576978a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:10:33.020630 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac39dee-9afa-4519-becc-6ab5291c084e service neutron] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:10:33.020879 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac39dee-9afa-4519-becc-6ab5291c084e service neutron] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:10:33.021134 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac39dee-9afa-4519-becc-6ab5291c084e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:10:33.032850 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac39dee-9afa-4519-becc-6ab5291c084e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:10:33.035016 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac39dee-9afa-4519-becc-6ab5291c084e service neutron] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:10:33.045591 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 732/1452] 10.222.0.23 () {60 vars in 1365 bytes} [Thu May 19 12:10:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:10:35.405801 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adf9555-806c-4a56-a036-df6a7fdff602 None None] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:10:35.406400 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adf9555-806c-4a56-a036-df6a7fdff602 None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:10:35.406400 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adf9555-806c-4a56-a036-df6a7fdff602 None None] PATH_INFO: `/` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:10:35.407718 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 721/1453] 127.0.0.1 () {54 vars in 916 bytes} [Thu May 19 12:10:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 12:10:35.413454 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f74fbd3-fbaa-4574-92ba-46a2c97a0de4 None None] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:10:35.413918 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f74fbd3-fbaa-4574-92ba-46a2c97a0de4 None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:10:35.414024 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f74fbd3-fbaa-4574-92ba-46a2c97a0de4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:10:35.432401 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.auth.core [None req-4f74fbd3-fbaa-4574-92ba-46a2c97a0de4 None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:10:35.433846 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-4f74fbd3-fbaa-4574-92ba-46a2c97a0de4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:10:35.442564 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 733/1454] 127.0.0.1 () {58 vars in 1013 bytes} [Thu May 19 12:10:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:10:35.480090 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5adf9555-806c-4a56-a036-df6a7fdff602 None None] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:10:35.491617 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5adf9555-806c-4a56-a036-df6a7fdff602 None None] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:10:35.495185 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6817d8-ab24-40e4-8eed-63ba25e19982 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a00b3fa7497456895e5e0bf105c4fc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd70d25bf69314820b8aaecb65576978a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:10:35.495853 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6817d8-ab24-40e4-8eed-63ba25e19982 service neutron] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:10:35.496142 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6817d8-ab24-40e4-8eed-63ba25e19982 service neutron] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:10:35.496419 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6817d8-ab24-40e4-8eed-63ba25e19982 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:10:35.506601 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6817d8-ab24-40e4-8eed-63ba25e19982 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:10:35.508078 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6817d8-ab24-40e4-8eed-63ba25e19982 service neutron] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:10:35.517840 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 722/1455] 10.222.0.23 () {60 vars in 1365 bytes} [Thu May 19 12:10:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:10:37.939341 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830147e6-b38e-4ab6-901b-99ab5c94f67b None None] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:10:37.939962 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830147e6-b38e-4ab6-901b-99ab5c94f67b None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:10:37.939962 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830147e6-b38e-4ab6-901b-99ab5c94f67b None None] PATH_INFO: `/` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:10:37.941371 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 734/1456] 127.0.0.1 () {54 vars in 916 bytes} [Thu May 19 12:10:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 12:10:37.946746 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e970eb-3d19-4e4e-b40d-48e6f85942d8 None None] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:10:37.948658 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e970eb-3d19-4e4e-b40d-48e6f85942d8 None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:10:37.948658 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e970eb-3d19-4e4e-b40d-48e6f85942d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:10:37.954392 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=70 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:10:37.966441 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.auth.core [None req-53e970eb-3d19-4e4e-b40d-48e6f85942d8 None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:10:37.967731 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-53e970eb-3d19-4e4e-b40d-48e6f85942d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:10:37.975563 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 723/1457] 127.0.0.1 () {58 vars in 1013 bytes} [Thu May 19 12:10:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:10:38.014364 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830147e6-b38e-4ab6-901b-99ab5c94f67b None None] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:10:38.020195 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=64 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:10:38.026253 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830147e6-b38e-4ab6-901b-99ab5c94f67b None None] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:10:38.030419 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d2deda-4ee5-4c29-917e-bb43d4a50d20 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a00b3fa7497456895e5e0bf105c4fc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd70d25bf69314820b8aaecb65576978a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:10:38.036939 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d2deda-4ee5-4c29-917e-bb43d4a50d20 service neutron] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:10:38.036939 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d2deda-4ee5-4c29-917e-bb43d4a50d20 service neutron] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:10:38.036939 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d2deda-4ee5-4c29-917e-bb43d4a50d20 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:10:38.041059 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d2deda-4ee5-4c29-917e-bb43d4a50d20 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:10:38.042505 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d2deda-4ee5-4c29-917e-bb43d4a50d20 service neutron] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:10:38.052137 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 735/1458] 10.222.0.23 () {60 vars in 1365 bytes} [Thu May 19 12:10:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:10:40.344018 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee311fc1-c28d-49a0-b635-3bae99c1b4ec None None] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:10:40.344617 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee311fc1-c28d-49a0-b635-3bae99c1b4ec None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:10:40.344617 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee311fc1-c28d-49a0-b635-3bae99c1b4ec None None] PATH_INFO: `/` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:10:40.345983 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 724/1459] 127.0.0.1 () {54 vars in 916 bytes} [Thu May 19 12:10:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 12:10:40.351842 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9881f1-d85a-4acc-89d0-76b182e184d3 None None] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:10:40.352305 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9881f1-d85a-4acc-89d0-76b182e184d3 None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:10:40.352400 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9881f1-d85a-4acc-89d0-76b182e184d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:10:40.371656 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.auth.core [None req-ae9881f1-d85a-4acc-89d0-76b182e184d3 None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:10:40.372902 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-ae9881f1-d85a-4acc-89d0-76b182e184d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:10:40.380885 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 736/1460] 127.0.0.1 () {58 vars in 1013 bytes} [Thu May 19 12:10:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:10:40.417674 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee311fc1-c28d-49a0-b635-3bae99c1b4ec None None] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:10:40.429354 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee311fc1-c28d-49a0-b635-3bae99c1b4ec None None] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:10:40.432839 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3772e2c7-c1b6-4205-a41d-2430ee325704 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a00b3fa7497456895e5e0bf105c4fc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd70d25bf69314820b8aaecb65576978a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:10:40.433471 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3772e2c7-c1b6-4205-a41d-2430ee325704 service neutron] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:10:40.433752 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3772e2c7-c1b6-4205-a41d-2430ee325704 service neutron] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:10:40.434098 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3772e2c7-c1b6-4205-a41d-2430ee325704 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:10:40.443081 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3772e2c7-c1b6-4205-a41d-2430ee325704 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:10:40.444517 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3772e2c7-c1b6-4205-a41d-2430ee325704 service neutron] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:10:40.454347 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 725/1461] 10.222.0.23 () {60 vars in 1365 bytes} [Thu May 19 12:10:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:10:42.797196 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e13f989-c5e0-46c8-a169-9cab43ee3587 None None] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:10:42.797768 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e13f989-c5e0-46c8-a169-9cab43ee3587 None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:10:42.797768 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e13f989-c5e0-46c8-a169-9cab43ee3587 None None] PATH_INFO: `/` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:10:42.799061 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 737/1462] 127.0.0.1 () {54 vars in 916 bytes} [Thu May 19 12:10:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 12:10:42.804666 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4921a6-bfd2-4882-9f40-eeb926ceb514 None None] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:10:42.805137 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4921a6-bfd2-4882-9f40-eeb926ceb514 None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:10:42.805250 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4921a6-bfd2-4882-9f40-eeb926ceb514 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:10:42.823426 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.auth.core [None req-5c4921a6-bfd2-4882-9f40-eeb926ceb514 None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:10:42.824669 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-5c4921a6-bfd2-4882-9f40-eeb926ceb514 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:10:42.833542 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 726/1463] 127.0.0.1 () {58 vars in 1013 bytes} [Thu May 19 12:10:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:10:42.870857 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e13f989-c5e0-46c8-a169-9cab43ee3587 None None] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:10:42.882671 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e13f989-c5e0-46c8-a169-9cab43ee3587 None None] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:10:42.886172 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b27193a-fcdc-4442-b245-957d12db38cd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a00b3fa7497456895e5e0bf105c4fc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd70d25bf69314820b8aaecb65576978a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:10:42.886802 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b27193a-fcdc-4442-b245-957d12db38cd service neutron] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:10:42.887085 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b27193a-fcdc-4442-b245-957d12db38cd service neutron] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:10:42.887363 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b27193a-fcdc-4442-b245-957d12db38cd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:10:42.897147 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b27193a-fcdc-4442-b245-957d12db38cd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:10:42.898614 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b27193a-fcdc-4442-b245-957d12db38cd service neutron] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:10:42.908706 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 738/1464] 10.222.0.23 () {60 vars in 1365 bytes} [Thu May 19 12:10:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:10:45.163585 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa22e74f-a67a-4dba-a4f9-8551bc7f127b None None] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:10:45.164181 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa22e74f-a67a-4dba-a4f9-8551bc7f127b None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:10:45.164181 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa22e74f-a67a-4dba-a4f9-8551bc7f127b None None] PATH_INFO: `/` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:10:45.165559 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 727/1465] 127.0.0.1 () {54 vars in 916 bytes} [Thu May 19 12:10:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 12:10:45.172470 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5755bca9-69d2-44f4-a8ca-fee527e515c5 None None] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:10:45.173041 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5755bca9-69d2-44f4-a8ca-fee527e515c5 None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:10:45.173041 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5755bca9-69d2-44f4-a8ca-fee527e515c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:10:45.192481 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.auth.core [None req-5755bca9-69d2-44f4-a8ca-fee527e515c5 None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:10:45.193703 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-5755bca9-69d2-44f4-a8ca-fee527e515c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:10:45.203672 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 739/1466] 127.0.0.1 () {58 vars in 1013 bytes} [Thu May 19 12:10:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:10:45.242053 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa22e74f-a67a-4dba-a4f9-8551bc7f127b None None] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:10:45.254366 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa22e74f-a67a-4dba-a4f9-8551bc7f127b None None] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:10:45.258034 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df3a5eb-116f-4c54-ade5-9051d280ce01 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a00b3fa7497456895e5e0bf105c4fc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd70d25bf69314820b8aaecb65576978a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:10:45.258693 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df3a5eb-116f-4c54-ade5-9051d280ce01 service neutron] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:10:45.258978 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df3a5eb-116f-4c54-ade5-9051d280ce01 service neutron] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:10:45.259251 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df3a5eb-116f-4c54-ade5-9051d280ce01 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:10:45.269557 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df3a5eb-116f-4c54-ade5-9051d280ce01 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:10:45.271016 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df3a5eb-116f-4c54-ade5-9051d280ce01 service neutron] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:10:45.283011 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 728/1467] 10.222.0.23 () {60 vars in 1365 bytes} [Thu May 19 12:10:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:10:47.549003 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c982563d-64d5-49e9-a493-823b72b09e93 None None] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:10:47.549572 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c982563d-64d5-49e9-a493-823b72b09e93 None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:10:47.549727 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c982563d-64d5-49e9-a493-823b72b09e93 None None] PATH_INFO: `/` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:10:47.551301 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 740/1468] 127.0.0.1 () {54 vars in 916 bytes} [Thu May 19 12:10:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 12:10:47.556714 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f095f4a6-bc0b-4768-94ab-240980bca2d1 None None] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:10:47.557149 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f095f4a6-bc0b-4768-94ab-240980bca2d1 None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:10:47.557457 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f095f4a6-bc0b-4768-94ab-240980bca2d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:10:47.575428 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.auth.core [None req-f095f4a6-bc0b-4768-94ab-240980bca2d1 None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:10:47.577226 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-f095f4a6-bc0b-4768-94ab-240980bca2d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:10:47.584979 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 729/1469] 127.0.0.1 () {58 vars in 1013 bytes} [Thu May 19 12:10:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:10:47.622291 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c982563d-64d5-49e9-a493-823b72b09e93 None None] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:10:47.633012 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c982563d-64d5-49e9-a493-823b72b09e93 None None] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:10:47.636355 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a733414d-da3f-48de-af44-678bf7fd197a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a00b3fa7497456895e5e0bf105c4fc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd70d25bf69314820b8aaecb65576978a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:10:47.636978 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a733414d-da3f-48de-af44-678bf7fd197a service neutron] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:10:47.637259 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a733414d-da3f-48de-af44-678bf7fd197a service neutron] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:10:47.637529 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a733414d-da3f-48de-af44-678bf7fd197a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:10:47.647079 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a733414d-da3f-48de-af44-678bf7fd197a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:10:47.648693 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a733414d-da3f-48de-af44-678bf7fd197a service neutron] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:10:47.658739 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 741/1470] 10.222.0.23 () {60 vars in 1365 bytes} [Thu May 19 12:10:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:10:57.570592 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=20 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:10:57.653905 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=23 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:11:21.291892 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8520d5-a437-45d4-9784-0b7b4b757748 None None] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:11:21.292464 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8520d5-a437-45d4-9784-0b7b4b757748 None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:11:21.292464 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8520d5-a437-45d4-9784-0b7b4b757748 None None] PATH_INFO: `/` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:11:21.293725 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 730/1471] 10.222.0.23 () {54 vars in 926 bytes} [Thu May 19 12:11:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 12:11:21.299474 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7266eb6f-4df6-42ff-a8e8-5c683194c47f None None] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:11:21.299966 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7266eb6f-4df6-42ff-a8e8-5c683194c47f None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:11:21.300093 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7266eb6f-4df6-42ff-a8e8-5c683194c47f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:11:21.321603 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.auth.core [None req-7266eb6f-4df6-42ff-a8e8-5c683194c47f None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:11:21.322834 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-7266eb6f-4df6-42ff-a8e8-5c683194c47f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:11:21.335946 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 742/1472] 10.222.0.23 () {58 vars in 1023 bytes} [Thu May 19 12:11:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:11:21.381101 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7966a699-b7fc-4a1f-a43b-f5155c2810f2 None None] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:11:21.381597 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7966a699-b7fc-4a1f-a43b-f5155c2810f2 None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:11:21.381879 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7966a699-b7fc-4a1f-a43b-f5155c2810f2 None None] PATH_INFO: `/` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:11:21.383233 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 731/1473] 10.222.0.23 () {54 vars in 962 bytes} [Thu May 19 12:11:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 12:11:21.388189 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de186e9a-258d-47e9-92d9-2a7d7593c45c None None] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:11:21.388486 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de186e9a-258d-47e9-92d9-2a7d7593c45c None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:11:21.389819 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de186e9a-258d-47e9-92d9-2a7d7593c45c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:11:21.409088 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.auth.core [None req-de186e9a-258d-47e9-92d9-2a7d7593c45c None None] MFA Rules not processed for user `21697802f2be42f9baa8e07490d5ebdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:11:21.410430 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-de186e9a-258d-47e9-92d9-2a7d7593c45c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:11:21.420247 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 743/1474] 10.222.0.23 () {58 vars in 1059 bytes} [Thu May 19 12:11:21 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:11:21.425032 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7966a699-b7fc-4a1f-a43b-f5155c2810f2 None None] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:11:21.436912 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7966a699-b7fc-4a1f-a43b-f5155c2810f2 None None] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:11:21.440571 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0171fa74-0082-45d5-9329-b243395237bc service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '21697802f2be42f9baa8e07490d5ebdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd70d25bf69314820b8aaecb65576978a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:11:21.441282 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0171fa74-0082-45d5-9329-b243395237bc service glance] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:11:21.441628 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0171fa74-0082-45d5-9329-b243395237bc service glance] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:11:21.441968 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0171fa74-0082-45d5-9329-b243395237bc service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:11:21.451765 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0171fa74-0082-45d5-9329-b243395237bc service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:11:21.453335 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0171fa74-0082-45d5-9329-b243395237bc service glance] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:11:21.462535 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 732/1475] 10.222.0.23 () {60 vars in 1365 bytes} [Thu May 19 12:11:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:11:23.836339 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd1bf7f-af43-4096-85eb-172259929229 None None] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:11:23.836920 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd1bf7f-af43-4096-85eb-172259929229 None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:11:23.836920 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd1bf7f-af43-4096-85eb-172259929229 None None] PATH_INFO: `/` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:11:23.838551 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 744/1476] 10.222.0.23 () {54 vars in 926 bytes} [Thu May 19 12:11:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 12:11:23.844161 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139e4a7b-066e-4a35-81ae-b0f897f6d62b None None] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:11:23.844587 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139e4a7b-066e-4a35-81ae-b0f897f6d62b None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:11:23.844723 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139e4a7b-066e-4a35-81ae-b0f897f6d62b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:11:23.863248 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.auth.core [None req-139e4a7b-066e-4a35-81ae-b0f897f6d62b None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:11:23.864472 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-139e4a7b-066e-4a35-81ae-b0f897f6d62b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:11:23.872320 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 733/1477] 10.222.0.23 () {58 vars in 1023 bytes} [Thu May 19 12:11:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:11:23.896504 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd1bf7f-af43-4096-85eb-172259929229 None None] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:11:23.909196 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd1bf7f-af43-4096-85eb-172259929229 None None] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:11:23.913616 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45e4ff6-5d44-4b5a-9013-57dff2519e3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:11:23.914259 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45e4ff6-5d44-4b5a-9013-57dff2519e3e admin admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:11:23.914547 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45e4ff6-5d44-4b5a-9013-57dff2519e3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:11:23.914824 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45e4ff6-5d44-4b5a-9013-57dff2519e3e admin admin] PATH_INFO: `/v3/projects` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:11:23.915560 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45e4ff6-5d44-4b5a-9013-57dff2519e3e admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:11:23.917344 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: WARNING py.warnings [None req-a45e4ff6-5d44-4b5a-9013-57dff2519e3e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 19 12:11:23.917344 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: warnings.warn(msg) May 19 12:11:23.917344 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: May 19 12:11:23.918487 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45e4ff6-5d44-4b5a-9013-57dff2519e3e admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:11:23.938711 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 745/1478] 10.222.0.23 () {56 vars in 1108 bytes} [Thu May 19 12:11:23 2022] GET /identity/v3/projects => generated 2553 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 12:11:26.213604 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353be69b-7488-4cbf-9935-7f0bdea165b5 None None] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:11:26.214143 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353be69b-7488-4cbf-9935-7f0bdea165b5 None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:11:26.214143 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353be69b-7488-4cbf-9935-7f0bdea165b5 None None] PATH_INFO: `/` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:11:26.215465 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 734/1479] 10.222.0.23 () {54 vars in 926 bytes} [Thu May 19 12:11:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 12:11:26.221325 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918ca793-0f5b-43f5-b735-27226cc76b43 None None] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:11:26.221793 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918ca793-0f5b-43f5-b735-27226cc76b43 None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:11:26.221915 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918ca793-0f5b-43f5-b735-27226cc76b43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:11:26.240589 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.auth.core [None req-918ca793-0f5b-43f5-b735-27226cc76b43 None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:11:26.241841 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-918ca793-0f5b-43f5-b735-27226cc76b43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:11:26.249882 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 746/1480] 10.222.0.23 () {58 vars in 1023 bytes} [Thu May 19 12:11:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:11:26.293034 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353be69b-7488-4cbf-9935-7f0bdea165b5 None None] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:11:26.307570 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353be69b-7488-4cbf-9935-7f0bdea165b5 None None] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:11:26.313361 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473cd1bc-34d0-4914-aade-f257aba4e1c7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34a98d75d84b4c67981377e0f26eca62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd70d25bf69314820b8aaecb65576978a', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:11:26.313991 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473cd1bc-34d0-4914-aade-f257aba4e1c7 service nova] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:11:26.314272 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473cd1bc-34d0-4914-aade-f257aba4e1c7 service nova] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:11:26.314544 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473cd1bc-34d0-4914-aade-f257aba4e1c7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:11:26.322765 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473cd1bc-34d0-4914-aade-f257aba4e1c7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:11:26.324421 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473cd1bc-34d0-4914-aade-f257aba4e1c7 service nova] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:11:26.334688 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 735/1481] 10.222.0.23 () {60 vars in 1365 bytes} [Thu May 19 12:11:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:11:28.763665 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba9cb98-cbb1-4a83-a95f-8b0fa8e3daf3 None None] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:11:28.764315 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba9cb98-cbb1-4a83-a95f-8b0fa8e3daf3 None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:11:28.764315 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba9cb98-cbb1-4a83-a95f-8b0fa8e3daf3 None None] PATH_INFO: `/` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:11:28.765695 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 747/1482] 10.222.0.23 () {54 vars in 926 bytes} [Thu May 19 12:11:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 12:11:28.770965 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d6b665-8f0c-4063-a49d-f692e889403b None None] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:11:28.771416 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d6b665-8f0c-4063-a49d-f692e889403b None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:11:28.771514 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d6b665-8f0c-4063-a49d-f692e889403b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:11:28.789081 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.auth.core [None req-04d6b665-8f0c-4063-a49d-f692e889403b None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:11:28.790257 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-04d6b665-8f0c-4063-a49d-f692e889403b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:11:28.798583 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 736/1483] 10.222.0.23 () {58 vars in 1023 bytes} [Thu May 19 12:11:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:11:28.834480 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba9cb98-cbb1-4a83-a95f-8b0fa8e3daf3 None None] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:11:28.845680 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba9cb98-cbb1-4a83-a95f-8b0fa8e3daf3 None None] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:11:28.849212 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34aba913-021b-4aa0-a733-1ada0326664b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a00b3fa7497456895e5e0bf105c4fc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd70d25bf69314820b8aaecb65576978a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:11:28.849849 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34aba913-021b-4aa0-a733-1ada0326664b service neutron] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:11:28.850140 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34aba913-021b-4aa0-a733-1ada0326664b service neutron] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:11:28.850428 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34aba913-021b-4aa0-a733-1ada0326664b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:11:28.860440 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34aba913-021b-4aa0-a733-1ada0326664b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:11:28.861982 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34aba913-021b-4aa0-a733-1ada0326664b service neutron] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:11:28.871592 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 748/1484] 10.222.0.23 () {60 vars in 1365 bytes} [Thu May 19 12:11:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:11:31.199214 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b8437d-4908-492f-94ff-9ebff31fed2c None None] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:11:31.199798 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b8437d-4908-492f-94ff-9ebff31fed2c None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:11:31.199798 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b8437d-4908-492f-94ff-9ebff31fed2c None None] PATH_INFO: `/` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:11:31.201728 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 737/1485] 10.222.0.23 () {54 vars in 926 bytes} [Thu May 19 12:11:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 12:11:31.207380 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373e5345-e766-4309-bd25-00bf76e3cf95 None None] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:11:31.207885 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373e5345-e766-4309-bd25-00bf76e3cf95 None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:11:31.207999 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373e5345-e766-4309-bd25-00bf76e3cf95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:11:31.227107 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.auth.core [None req-373e5345-e766-4309-bd25-00bf76e3cf95 None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:11:31.228338 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-373e5345-e766-4309-bd25-00bf76e3cf95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:11:31.236497 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 749/1486] 10.222.0.23 () {58 vars in 1023 bytes} [Thu May 19 12:11:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:11:31.275134 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b8437d-4908-492f-94ff-9ebff31fed2c None None] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:11:31.286091 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b8437d-4908-492f-94ff-9ebff31fed2c None None] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:11:31.289615 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e855b29a-f774-4a0b-9b18-bdf9f334b1ae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a00b3fa7497456895e5e0bf105c4fc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd70d25bf69314820b8aaecb65576978a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:11:31.290244 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e855b29a-f774-4a0b-9b18-bdf9f334b1ae service neutron] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:11:31.290520 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e855b29a-f774-4a0b-9b18-bdf9f334b1ae service neutron] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:11:31.290789 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e855b29a-f774-4a0b-9b18-bdf9f334b1ae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:11:31.300022 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e855b29a-f774-4a0b-9b18-bdf9f334b1ae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:11:31.301492 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e855b29a-f774-4a0b-9b18-bdf9f334b1ae service neutron] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:11:31.311362 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 738/1487] 10.222.0.23 () {60 vars in 1365 bytes} [Thu May 19 12:11:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:11:33.737000 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cc819f-008b-43d8-b76c-80a6ec2a2e32 None None] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:11:33.737562 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cc819f-008b-43d8-b76c-80a6ec2a2e32 None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:11:33.737562 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cc819f-008b-43d8-b76c-80a6ec2a2e32 None None] PATH_INFO: `/` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:11:33.738948 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 750/1488] 10.222.0.23 () {54 vars in 926 bytes} [Thu May 19 12:11:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 12:11:33.744206 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77dd9af-edf7-4687-840a-d4ee5f8e7574 None None] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:11:33.744633 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77dd9af-edf7-4687-840a-d4ee5f8e7574 None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:11:33.744764 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77dd9af-edf7-4687-840a-d4ee5f8e7574 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:11:33.762914 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.auth.core [None req-c77dd9af-edf7-4687-840a-d4ee5f8e7574 None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:11:33.764836 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-c77dd9af-edf7-4687-840a-d4ee5f8e7574 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:11:33.772577 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 739/1489] 10.222.0.23 () {58 vars in 1023 bytes} [Thu May 19 12:11:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:11:33.813857 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56cc819f-008b-43d8-b76c-80a6ec2a2e32 None None] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:11:33.824714 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56cc819f-008b-43d8-b76c-80a6ec2a2e32 None None] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:11:33.828467 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dabe1f4-e26a-4fcb-af35-dcba7529c5f9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a00b3fa7497456895e5e0bf105c4fc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd70d25bf69314820b8aaecb65576978a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:11:33.829057 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dabe1f4-e26a-4fcb-af35-dcba7529c5f9 service neutron] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:11:33.829397 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dabe1f4-e26a-4fcb-af35-dcba7529c5f9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:11:33.829621 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dabe1f4-e26a-4fcb-af35-dcba7529c5f9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:11:33.839112 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dabe1f4-e26a-4fcb-af35-dcba7529c5f9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:11:33.840580 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dabe1f4-e26a-4fcb-af35-dcba7529c5f9 service neutron] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:11:33.850023 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 751/1490] 10.222.0.23 () {60 vars in 1365 bytes} [Thu May 19 12:11:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:11:36.669109 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276a7fed-b3b4-4530-bd96-468eb7fcbf64 None None] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:11:36.669700 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276a7fed-b3b4-4530-bd96-468eb7fcbf64 None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:11:36.669700 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276a7fed-b3b4-4530-bd96-468eb7fcbf64 None None] PATH_INFO: `/` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:11:36.671777 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 740/1491] 10.222.0.23 () {54 vars in 926 bytes} [Thu May 19 12:11:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 12:11:36.677348 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0171c59e-06e0-4055-bfe8-c91db1a0b0ed None None] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:11:36.677815 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0171c59e-06e0-4055-bfe8-c91db1a0b0ed None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:11:36.677927 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0171c59e-06e0-4055-bfe8-c91db1a0b0ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:11:36.697237 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.auth.core [None req-0171c59e-06e0-4055-bfe8-c91db1a0b0ed None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:11:36.698431 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-0171c59e-06e0-4055-bfe8-c91db1a0b0ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:11:36.706893 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 752/1492] 10.222.0.23 () {58 vars in 1023 bytes} [Thu May 19 12:11:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:11:36.747185 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276a7fed-b3b4-4530-bd96-468eb7fcbf64 None None] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:11:36.758743 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276a7fed-b3b4-4530-bd96-468eb7fcbf64 None None] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:11:36.762250 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3966a730-bbfb-4f81-b948-83f3b9a45d91 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a00b3fa7497456895e5e0bf105c4fc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd70d25bf69314820b8aaecb65576978a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:11:36.762879 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3966a730-bbfb-4f81-b948-83f3b9a45d91 service neutron] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:11:36.763157 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3966a730-bbfb-4f81-b948-83f3b9a45d91 service neutron] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:11:36.763436 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3966a730-bbfb-4f81-b948-83f3b9a45d91 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:11:36.773018 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3966a730-bbfb-4f81-b948-83f3b9a45d91 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:11:36.774442 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3966a730-bbfb-4f81-b948-83f3b9a45d91 service neutron] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:11:36.784059 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 741/1493] 10.222.0.23 () {60 vars in 1365 bytes} [Thu May 19 12:11:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:11:46.691289 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=33 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:11:46.779537 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=33 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:13:51.707348 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb6c51a-d10b-46fe-aabc-1155393eb1ff None None] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:13:51.707979 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb6c51a-d10b-46fe-aabc-1155393eb1ff None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:13:51.707979 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb6c51a-d10b-46fe-aabc-1155393eb1ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:13:51.727128 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.auth.core [None req-8fb6c51a-d10b-46fe-aabc-1155393eb1ff None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:13:51.728357 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-8fb6c51a-d10b-46fe-aabc-1155393eb1ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:13:51.736655 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 753/1494] 10.222.0.23 () {58 vars in 957 bytes} [Thu May 19 12:13:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:13:51.746123 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c1715c-575d-4dd0-b184-e720c22ad094 None None] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:13:51.757852 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c1715c-575d-4dd0-b184-e720c22ad094 None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:13:51.757852 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c1715c-575d-4dd0-b184-e720c22ad094 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:13:51.764990 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.auth.core [None req-f8c1715c-575d-4dd0-b184-e720c22ad094 None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:13:51.766179 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-f8c1715c-575d-4dd0-b184-e720c22ad094 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:13:51.773865 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 742/1495] 10.222.0.23 () {58 vars in 957 bytes} [Thu May 19 12:13:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:13:51.779419 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb6c51a-d10b-46fe-aabc-1155393eb1ff None None] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:13:51.789289 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb6c51a-d10b-46fe-aabc-1155393eb1ff None None] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:13:51.795010 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27398088-de1d-4aad-ba73-32e0b19d6b37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:13:51.795710 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27398088-de1d-4aad-ba73-32e0b19d6b37 admin admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:13:51.796028 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27398088-de1d-4aad-ba73-32e0b19d6b37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:13:51.796309 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27398088-de1d-4aad-ba73-32e0b19d6b37 admin admin] PATH_INFO: `/v3/domains` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:13:51.797069 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27398088-de1d-4aad-ba73-32e0b19d6b37 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:13:51.797378 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27398088-de1d-4aad-ba73-32e0b19d6b37 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:13:51.798442 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: WARNING py.warnings [None req-27398088-de1d-4aad-ba73-32e0b19d6b37 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 19 12:13:51.798442 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: warnings.warn(msg) May 19 12:13:51.798442 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: May 19 12:13:51.799370 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27398088-de1d-4aad-ba73-32e0b19d6b37 admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:13:51.814231 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 754/1496] 10.222.0.23 () {58 vars in 1152 bytes} [Thu May 19 12:13:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:13:51.823121 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c1715c-575d-4dd0-b184-e720c22ad094 None None] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:13:51.833587 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c1715c-575d-4dd0-b184-e720c22ad094 None None] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:13:51.838131 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a545d6a6-5da8-4d9a-b868-f1c8362bdb90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:13:51.838786 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a545d6a6-5da8-4d9a-b868-f1c8362bdb90 admin admin] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:13:51.840367 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a545d6a6-5da8-4d9a-b868-f1c8362bdb90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:13:51.840658 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a545d6a6-5da8-4d9a-b868-f1c8362bdb90 admin admin] PATH_INFO: `/v3/projects` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:13:51.841406 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a545d6a6-5da8-4d9a-b868-f1c8362bdb90 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:13:51.843458 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: WARNING py.warnings [None req-a545d6a6-5da8-4d9a-b868-f1c8362bdb90 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 19 12:13:51.843458 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: warnings.warn(msg) May 19 12:13:51.843458 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: May 19 12:13:51.844742 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a545d6a6-5da8-4d9a-b868-f1c8362bdb90 admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:13:51.979547 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 743/1497] 10.222.0.23 () {60 vars in 1152 bytes} [Thu May 19 12:13:51 2022] POST /identity/v3/projects => generated 377 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 12:13:51.983699 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27398088-de1d-4aad-ba73-32e0b19d6b37 admin admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:13:51.995748 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27398088-de1d-4aad-ba73-32e0b19d6b37 admin admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:13:52.051882 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80cb639c-55d2-4bb5-8693-7f7710fc368c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:13:52.052552 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cb639c-55d2-4bb5-8693-7f7710fc368c admin admin] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:13:52.052836 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cb639c-55d2-4bb5-8693-7f7710fc368c admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:13:52.053117 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cb639c-55d2-4bb5-8693-7f7710fc368c admin admin] PATH_INFO: `/v3/users` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:13:52.053800 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80cb639c-55d2-4bb5-8693-7f7710fc368c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:13:52.054833 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: WARNING py.warnings [None req-80cb639c-55d2-4bb5-8693-7f7710fc368c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 19 12:13:52.054833 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: warnings.warn(msg) May 19 12:13:52.054833 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: May 19 12:13:52.056103 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80cb639c-55d2-4bb5-8693-7f7710fc368c admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:13:52.172127 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 755/1498] 10.222.0.23 () {60 vars in 1143 bytes} [Thu May 19 12:13:51 2022] POST /identity/v3/users => generated 289 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 12:13:52.176200 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a545d6a6-5da8-4d9a-b868-f1c8362bdb90 admin admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:13:52.187324 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a545d6a6-5da8-4d9a-b868-f1c8362bdb90 admin admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:13:52.191988 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6aef640-9d1e-446f-bab6-a04e165dc14c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:13:52.192637 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6aef640-9d1e-446f-bab6-a04e165dc14c admin admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:13:52.192922 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6aef640-9d1e-446f-bab6-a04e165dc14c admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:13:52.193198 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6aef640-9d1e-446f-bab6-a04e165dc14c admin admin] PATH_INFO: `/v3/roles` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:13:52.193924 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6aef640-9d1e-446f-bab6-a04e165dc14c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:13:52.194938 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: WARNING py.warnings [None req-e6aef640-9d1e-446f-bab6-a04e165dc14c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 19 12:13:52.194938 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: warnings.warn(msg) May 19 12:13:52.194938 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: May 19 12:13:52.195921 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6aef640-9d1e-446f-bab6-a04e165dc14c admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:13:52.209240 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 744/1499] 10.222.0.23 () {58 vars in 1121 bytes} [Thu May 19 12:13:52 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 12:13:52.213569 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80cb639c-55d2-4bb5-8693-7f7710fc368c admin admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:13:52.223858 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80cb639c-55d2-4bb5-8693-7f7710fc368c admin admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:13:52.228360 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5811d2-bc32-4438-abb1-fccea6bf8439 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:13:52.229012 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5811d2-bc32-4438-abb1-fccea6bf8439 admin admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:13:52.229312 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5811d2-bc32-4438-abb1-fccea6bf8439 admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:13:52.230177 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5811d2-bc32-4438-abb1-fccea6bf8439 admin admin] PATH_INFO: `/v3/roles` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:13:52.230937 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5811d2-bc32-4438-abb1-fccea6bf8439 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:13:52.231991 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: WARNING py.warnings [None req-7f5811d2-bc32-4438-abb1-fccea6bf8439 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 19 12:13:52.231991 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: warnings.warn(msg) May 19 12:13:52.231991 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: May 19 12:13:52.232924 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5811d2-bc32-4438-abb1-fccea6bf8439 admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:13:52.247650 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 756/1500] 10.222.0.23 () {58 vars in 1121 bytes} [Thu May 19 12:13:52 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 12:13:52.251815 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6aef640-9d1e-446f-bab6-a04e165dc14c admin admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:13:52.261985 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6aef640-9d1e-446f-bab6-a04e165dc14c admin admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:13:52.266580 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1639af32-eb50-4555-b551-7f5489cfce11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:13:52.269252 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1639af32-eb50-4555-b551-7f5489cfce11 admin admin] REQUEST_METHOD: `PUT` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:13:52.269252 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1639af32-eb50-4555-b551-7f5489cfce11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:13:52.269252 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1639af32-eb50-4555-b551-7f5489cfce11 admin admin] PATH_INFO: `/v3/projects/90ce461a8f834c69bb949ea920495a44/users/4596d6d2663a477d9b971f3fb0a44765/roles/259eb400cdb4439db72d14456ae9cd08` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:13:52.282931 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1639af32-eb50-4555-b551-7f5489cfce11 admin admin] RBAC: Authorizing `identity:create_grant(project_id=90ce461a8f834c69bb949ea920495a44, user_id=4596d6d2663a477d9b971f3fb0a44765, role_id=259eb400cdb4439db72d14456ae9cd08)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:13:52.284015 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: WARNING py.warnings [None req-1639af32-eb50-4555-b551-7f5489cfce11 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 19 12:13:52.284015 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: warnings.warn(msg) May 19 12:13:52.284015 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: May 19 12:13:52.285967 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1639af32-eb50-4555-b551-7f5489cfce11 admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:13:52.355515 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 745/1501] 10.222.0.23 () {60 vars in 1482 bytes} [Thu May 19 12:13:52 2022] PUT /identity/v3/projects/90ce461a8f834c69bb949ea920495a44/users/4596d6d2663a477d9b971f3fb0a44765/roles/259eb400cdb4439db72d14456ae9cd08 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:13:52.366673 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4301e1-26cf-4370-aede-28bd98ef732f None None] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:13:52.366971 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4301e1-26cf-4370-aede-28bd98ef732f None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:13:52.367253 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4301e1-26cf-4370-aede-28bd98ef732f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:13:52.395647 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.auth.core [None req-4f4301e1-26cf-4370-aede-28bd98ef732f None None] MFA Rules not processed for user `4596d6d2663a477d9b971f3fb0a44765`. Rule list: `[]` (Enabled: `True`). {{(pid=80540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:13:52.396948 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-4f4301e1-26cf-4370-aede-28bd98ef732f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:13:52.461009 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 757/1502] 10.222.0.23 () {58 vars in 957 bytes} [Thu May 19 12:13:52 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:13:52.475182 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e099f2a1-ce66-46f8-955b-5ee5637f5c07 None None] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:13:52.475473 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e099f2a1-ce66-46f8-955b-5ee5637f5c07 None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:13:52.475755 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e099f2a1-ce66-46f8-955b-5ee5637f5c07 None None] PATH_INFO: `/` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:13:52.477134 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 746/1503] 10.222.0.23 () {54 vars in 963 bytes} [Thu May 19 12:13:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 12:13:52.482694 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824476a1-ba3c-4c42-8f58-379b29da0fa9 None None] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:13:52.482930 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824476a1-ba3c-4c42-8f58-379b29da0fa9 None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:13:52.483205 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824476a1-ba3c-4c42-8f58-379b29da0fa9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:13:52.502744 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.auth.core [None req-824476a1-ba3c-4c42-8f58-379b29da0fa9 None None] MFA Rules not processed for user `8800254c0e784e42a791fb9abc274d79`. Rule list: `[]` (Enabled: `True`). {{(pid=80540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:13:52.503990 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-824476a1-ba3c-4c42-8f58-379b29da0fa9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:13:52.548952 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:13:52.556484 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 758/1504] 10.222.0.23 () {58 vars in 1060 bytes} [Thu May 19 12:13:52 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:13:52.565198 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e099f2a1-ce66-46f8-955b-5ee5637f5c07 None None] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:13:52.576091 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e099f2a1-ce66-46f8-955b-5ee5637f5c07 None None] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:13:52.579527 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d532f411-7693-4a91-978c-2d63138e77f2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8800254c0e784e42a791fb9abc274d79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd70d25bf69314820b8aaecb65576978a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:13:52.580192 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d532f411-7693-4a91-978c-2d63138e77f2 service cinder] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:13:52.580474 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d532f411-7693-4a91-978c-2d63138e77f2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:13:52.580748 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d532f411-7693-4a91-978c-2d63138e77f2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:13:52.589413 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d532f411-7693-4a91-978c-2d63138e77f2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:13:52.592253 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d532f411-7693-4a91-978c-2d63138e77f2 service cinder] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:13:52.599965 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 747/1505] 10.222.0.23 () {60 vars in 1365 bytes} [Thu May 19 12:13:52 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:13:52.643629 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448d96c1-e7ea-489e-8d72-f26426102039 None None] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:13:52.644029 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448d96c1-e7ea-489e-8d72-f26426102039 None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:13:52.644406 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448d96c1-e7ea-489e-8d72-f26426102039 None None] PATH_INFO: `/` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:13:52.645699 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 759/1506] 10.222.0.23 () {52 vars in 829 bytes} [Thu May 19 12:13:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 12:13:52.649185 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d532f411-7693-4a91-978c-2d63138e77f2 service cinder] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:13:52.712268 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d532f411-7693-4a91-978c-2d63138e77f2 service cinder] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:13:52.717663 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d13d62-9f2b-4e85-a546-c539218e945b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:13:52.718525 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d13d62-9f2b-4e85-a546-c539218e945b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:13:52.718860 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d13d62-9f2b-4e85-a546-c539218e945b admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:13:52.719133 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d13d62-9f2b-4e85-a546-c539218e945b admin admin] PATH_INFO: `/v3/users/4596d6d2663a477d9b971f3fb0a44765` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:13:52.720380 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d13d62-9f2b-4e85-a546-c539218e945b admin admin] RBAC: Authorizing `identity:delete_user(user_id=4596d6d2663a477d9b971f3fb0a44765)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:13:52.721421 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: WARNING py.warnings [None req-f6d13d62-9f2b-4e85-a546-c539218e945b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 19 12:13:52.721421 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: warnings.warn(msg) May 19 12:13:52.721421 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: May 19 12:13:52.722633 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d13d62-9f2b-4e85-a546-c539218e945b admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:13:53.115184 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-f6d13d62-9f2b-4e85-a546-c539218e945b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4596d6d2663a477d9b971f3fb0a44765'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:13:53.134029 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-f6d13d62-9f2b-4e85-a546-c539218e945b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4596d6d2663a477d9b971f3fb0a44765'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:13:53.134818 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-f6d13d62-9f2b-4e85-a546-c539218e945b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4596d6d2663a477d9b971f3fb0a44765'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:13:53.234220 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-f6d13d62-9f2b-4e85-a546-c539218e945b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4596d6d2663a477d9b971f3fb0a44765'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:13:53.243514 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:13:53.248081 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-f6d13d62-9f2b-4e85-a546-c539218e945b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4596d6d2663a477d9b971f3fb0a44765'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:13:53.254164 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 748/1507] 10.222.0.23 () {58 vars in 1223 bytes} [Thu May 19 12:13:52 2022] DELETE /identity/v3/users/4596d6d2663a477d9b971f3fb0a44765 => generated 0 bytes in 605 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:13:53.258615 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f2db65-b530-4cf1-bf88-4c40624a4f70 None None] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:13:53.259141 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f2db65-b530-4cf1-bf88-4c40624a4f70 None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:13:53.259248 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f2db65-b530-4cf1-bf88-4c40624a4f70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:13:53.278885 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.auth.core [None req-f1f2db65-b530-4cf1-bf88-4c40624a4f70 None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:13:53.280128 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-f1f2db65-b530-4cf1-bf88-4c40624a4f70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:13:53.342793 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 760/1508] 10.222.0.23 () {58 vars in 957 bytes} [Thu May 19 12:13:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:13:53.352357 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d13d62-9f2b-4e85-a546-c539218e945b admin admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:13:53.354605 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-f6d13d62-9f2b-4e85-a546-c539218e945b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:13:53.413242 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d13d62-9f2b-4e85-a546-c539218e945b admin admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:13:53.416858 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95a63c3-605a-4a67-b940-9699d2399038 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a00b3fa7497456895e5e0bf105c4fc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd70d25bf69314820b8aaecb65576978a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:13:53.417641 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95a63c3-605a-4a67-b940-9699d2399038 service neutron] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:13:53.417840 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95a63c3-605a-4a67-b940-9699d2399038 service neutron] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:13:53.418114 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95a63c3-605a-4a67-b940-9699d2399038 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:13:53.427137 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95a63c3-605a-4a67-b940-9699d2399038 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:13:53.428595 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95a63c3-605a-4a67-b940-9699d2399038 service neutron] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:13:53.438607 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 749/1509] 10.222.0.23 () {60 vars in 1365 bytes} [Thu May 19 12:13:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:13:54.134837 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f2db65-b530-4cf1-bf88-4c40624a4f70 None None] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:13:54.139749 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-f1f2db65-b530-4cf1-bf88-4c40624a4f70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:13:54.150130 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f2db65-b530-4cf1-bf88-4c40624a4f70 None None] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:13:54.155307 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2320ba-c661-48f1-b1ab-448ddc0a697f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:13:54.156226 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2320ba-c661-48f1-b1ab-448ddc0a697f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:13:54.156516 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2320ba-c661-48f1-b1ab-448ddc0a697f admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:13:54.156861 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2320ba-c661-48f1-b1ab-448ddc0a697f admin admin] PATH_INFO: `/v3/projects/90ce461a8f834c69bb949ea920495a44` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:13:54.158025 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2320ba-c661-48f1-b1ab-448ddc0a697f admin admin] RBAC: Authorizing `identity:delete_project(project_id=90ce461a8f834c69bb949ea920495a44)` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:13:54.159803 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: WARNING py.warnings [None req-0c2320ba-c661-48f1-b1ab-448ddc0a697f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 19 12:13:54.159803 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: warnings.warn(msg) May 19 12:13:54.159803 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: May 19 12:13:54.160454 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2320ba-c661-48f1-b1ab-448ddc0a697f admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:13:54.303170 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-0c2320ba-c661-48f1-b1ab-448ddc0a697f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '90ce461a8f834c69bb949ea920495a44', 'request_id': 'req-0c2320ba-c661-48f1-b1ab-448ddc0a697f'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:13:54.318628 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-0c2320ba-c661-48f1-b1ab-448ddc0a697f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '90ce461a8f834c69bb949ea920495a44', 'request_id': 'req-0c2320ba-c661-48f1-b1ab-448ddc0a697f'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:13:54.330447 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-0c2320ba-c661-48f1-b1ab-448ddc0a697f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '90ce461a8f834c69bb949ea920495a44', 'request_id': 'req-0c2320ba-c661-48f1-b1ab-448ddc0a697f'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:13:54.457219 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-0c2320ba-c661-48f1-b1ab-448ddc0a697f admin admin] The token cache is being invalidate because project 90ce461a8f834c69bb949ea920495a44 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80540) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 12:13:54.457509 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-0c2320ba-c661-48f1-b1ab-448ddc0a697f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:13:54.458221 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-0c2320ba-c661-48f1-b1ab-448ddc0a697f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:13:54.459925 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 761/1510] 10.222.0.23 () {58 vars in 1232 bytes} [Thu May 19 12:13:54 2022] DELETE /identity/v3/projects/90ce461a8f834c69bb949ea920495a44 => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:14:03.434388 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=29 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:14:04.453906 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=55,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:15:38.538856 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde79512-7a20-4b68-bc3e-f52769080aa1 None None] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:15:38.539495 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde79512-7a20-4b68-bc3e-f52769080aa1 None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:15:38.539495 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde79512-7a20-4b68-bc3e-f52769080aa1 None None] PATH_INFO: `/` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:15:38.540912 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 750/1511] 127.0.0.1 () {54 vars in 901 bytes} [Thu May 19 12:15:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 12:15:38.546918 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc880a7e-7b63-477b-b1a3-ba21cceebea0 None None] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:15:38.547482 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc880a7e-7b63-477b-b1a3-ba21cceebea0 None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:15:38.547604 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc880a7e-7b63-477b-b1a3-ba21cceebea0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:15:38.567012 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.auth.core [None req-cc880a7e-7b63-477b-b1a3-ba21cceebea0 None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:15:38.568282 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-cc880a7e-7b63-477b-b1a3-ba21cceebea0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:15:38.633752 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 762/1512] 127.0.0.1 () {58 vars in 998 bytes} [Thu May 19 12:15:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:15:38.805749 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde79512-7a20-4b68-bc3e-f52769080aa1 None None] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:15:38.807486 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-dde79512-7a20-4b68-bc3e-f52769080aa1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:15:38.881881 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde79512-7a20-4b68-bc3e-f52769080aa1 None None] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:15:38.887530 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88489ae-b6fc-4c10-ae72-4e892154e7b3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34a98d75d84b4c67981377e0f26eca62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd70d25bf69314820b8aaecb65576978a', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:15:38.888207 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88489ae-b6fc-4c10-ae72-4e892154e7b3 service nova] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:15:38.888497 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88489ae-b6fc-4c10-ae72-4e892154e7b3 service nova] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:15:38.888775 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88489ae-b6fc-4c10-ae72-4e892154e7b3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:15:38.897015 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88489ae-b6fc-4c10-ae72-4e892154e7b3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:15:38.898623 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88489ae-b6fc-4c10-ae72-4e892154e7b3 service nova] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:15:38.908071 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 751/1513] 10.222.0.23 () {60 vars in 1365 bytes} [Thu May 19 12:15:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:15:39.820062 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afd0619-23ae-4f83-9de4-908f3bbaec98 None None] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:15:39.820604 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afd0619-23ae-4f83-9de4-908f3bbaec98 None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:15:39.820726 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afd0619-23ae-4f83-9de4-908f3bbaec98 None None] PATH_INFO: `/` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:15:39.822035 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 763/1514] 127.0.0.1 () {54 vars in 901 bytes} [Thu May 19 12:15:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 12:15:39.827380 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7096888d-697c-465e-9579-62dfdb329b9b None None] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:15:39.827900 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7096888d-697c-465e-9579-62dfdb329b9b None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:15:39.828003 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7096888d-697c-465e-9579-62dfdb329b9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:15:39.847247 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.auth.core [None req-7096888d-697c-465e-9579-62dfdb329b9b None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:15:39.848525 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-7096888d-697c-465e-9579-62dfdb329b9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:15:39.854284 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fdf376-15da-462e-bd6c-68498e4b8a7a None None] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:15:39.854579 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fdf376-15da-462e-bd6c-68498e4b8a7a None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:15:39.854919 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fdf376-15da-462e-bd6c-68498e4b8a7a None None] PATH_INFO: `/v3` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:15:39.856220 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 764/1515] 10.222.0.81 () {54 vars in 925 bytes} [Thu May 19 12:15:39 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 12:15:39.856970 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 752/1516] 127.0.0.1 () {58 vars in 998 bytes} [Thu May 19 12:15:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:15:39.862901 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461f5268-16a6-4c30-bc02-ba94534650ce None None] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:15:39.863194 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461f5268-16a6-4c30-bc02-ba94534650ce None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:15:39.863486 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461f5268-16a6-4c30-bc02-ba94534650ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:15:39.883056 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.auth.core [None req-461f5268-16a6-4c30-bc02-ba94534650ce None None] MFA Rules not processed for user `50f937eed4f3414fa6f19cab2a84eb5f`. Rule list: `[]` (Enabled: `True`). {{(pid=80540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:15:39.884286 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-461f5268-16a6-4c30-bc02-ba94534650ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:15:39.948501 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 765/1517] 10.222.0.81 () {58 vars in 1015 bytes} [Thu May 19 12:15:39 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:15:40.015743 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7096888d-697c-465e-9579-62dfdb329b9b None None] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:15:40.027918 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7096888d-697c-465e-9579-62dfdb329b9b None None] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:15:40.034243 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c676d2ff-a384-47ed-be30-0ee5b349827f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34a98d75d84b4c67981377e0f26eca62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd70d25bf69314820b8aaecb65576978a', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:15:40.034875 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c676d2ff-a384-47ed-be30-0ee5b349827f service nova] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:15:40.035158 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c676d2ff-a384-47ed-be30-0ee5b349827f service nova] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:15:40.035433 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c676d2ff-a384-47ed-be30-0ee5b349827f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:15:40.043496 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c676d2ff-a384-47ed-be30-0ee5b349827f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:15:40.045154 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c676d2ff-a384-47ed-be30-0ee5b349827f service nova] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:15:40.055192 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 753/1518] 10.222.0.23 () {60 vars in 1365 bytes} [Thu May 19 12:15:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:15:49.279298 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02f82fb-b801-4e1e-9685-b679e8491bb5 None None] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:15:49.279882 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02f82fb-b801-4e1e-9685-b679e8491bb5 None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:15:49.279882 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02f82fb-b801-4e1e-9685-b679e8491bb5 None None] PATH_INFO: `/` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:15:49.281256 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 766/1519] 10.222.0.23 () {54 vars in 960 bytes} [Thu May 19 12:15:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 12:15:49.286433 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d9b2b5-15ce-4c0a-8723-5463520554b8 None None] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:15:49.286841 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d9b2b5-15ce-4c0a-8723-5463520554b8 None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:15:49.287016 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d9b2b5-15ce-4c0a-8723-5463520554b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:15:49.305654 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.auth.core [None req-d3d9b2b5-15ce-4c0a-8723-5463520554b8 None None] MFA Rules not processed for user `50f937eed4f3414fa6f19cab2a84eb5f`. Rule list: `[]` (Enabled: `True`). {{(pid=80539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:15:49.306857 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-d3d9b2b5-15ce-4c0a-8723-5463520554b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:15:49.316150 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 754/1520] 10.222.0.23 () {58 vars in 1057 bytes} [Thu May 19 12:15:49 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:15:49.324053 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02f82fb-b801-4e1e-9685-b679e8491bb5 None None] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:15:49.337950 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02f82fb-b801-4e1e-9685-b679e8491bb5 None None] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:15:49.342998 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff27ab1-e3b2-4050-a3cd-34a9323d4e9d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50f937eed4f3414fa6f19cab2a84eb5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd70d25bf69314820b8aaecb65576978a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:15:49.343629 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff27ab1-e3b2-4050-a3cd-34a9323d4e9d service placement] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:15:49.344011 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff27ab1-e3b2-4050-a3cd-34a9323d4e9d service placement] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:15:49.350298 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff27ab1-e3b2-4050-a3cd-34a9323d4e9d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:15:49.352986 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff27ab1-e3b2-4050-a3cd-34a9323d4e9d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:15:49.354640 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff27ab1-e3b2-4050-a3cd-34a9323d4e9d service placement] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:15:49.364516 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 767/1521] 10.222.0.23 () {60 vars in 1365 bytes} [Thu May 19 12:15:49 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:15:51.381375 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011018bf-89cf-4017-8f05-c09228e475f8 None None] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:15:51.381945 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011018bf-89cf-4017-8f05-c09228e475f8 None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:15:51.381945 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011018bf-89cf-4017-8f05-c09228e475f8 None None] PATH_INFO: `/` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:15:51.383342 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 755/1522] 127.0.0.1 () {54 vars in 901 bytes} [Thu May 19 12:15:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 12:15:51.388224 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a98b4c-c9d4-4f7d-8ab9-1a4302bf648d None None] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:15:51.388679 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a98b4c-c9d4-4f7d-8ab9-1a4302bf648d None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:15:51.388786 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a98b4c-c9d4-4f7d-8ab9-1a4302bf648d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:15:51.406449 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.auth.core [None req-a3a98b4c-c9d4-4f7d-8ab9-1a4302bf648d None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:15:51.407630 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-a3a98b4c-c9d4-4f7d-8ab9-1a4302bf648d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:15:51.415880 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 768/1523] 127.0.0.1 () {58 vars in 998 bytes} [Thu May 19 12:15:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:15:51.570433 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011018bf-89cf-4017-8f05-c09228e475f8 None None] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:15:51.584835 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011018bf-89cf-4017-8f05-c09228e475f8 None None] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:15:51.590506 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428ea562-9eeb-49f5-b71c-9181ca70b63a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34a98d75d84b4c67981377e0f26eca62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd70d25bf69314820b8aaecb65576978a', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:15:51.591150 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428ea562-9eeb-49f5-b71c-9181ca70b63a service nova] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:15:51.591440 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428ea562-9eeb-49f5-b71c-9181ca70b63a service nova] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:15:51.591717 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428ea562-9eeb-49f5-b71c-9181ca70b63a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:15:51.601393 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428ea562-9eeb-49f5-b71c-9181ca70b63a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:15:51.602967 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428ea562-9eeb-49f5-b71c-9181ca70b63a service nova] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:15:51.613477 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 756/1524] 10.222.0.23 () {60 vars in 1365 bytes} [Thu May 19 12:15:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:15:52.508349 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d01fae-c1b8-45c5-b86d-c3e1c2df0089 None None] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:15:52.508904 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d01fae-c1b8-45c5-b86d-c3e1c2df0089 None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:15:52.508904 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d01fae-c1b8-45c5-b86d-c3e1c2df0089 None None] PATH_INFO: `/` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:15:52.510200 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 769/1525] 127.0.0.1 () {54 vars in 901 bytes} [Thu May 19 12:15:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 12:15:52.515128 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec5c101-38d4-4598-b640-bbf652ff6d99 None None] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:15:52.515555 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec5c101-38d4-4598-b640-bbf652ff6d99 None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:15:52.515722 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec5c101-38d4-4598-b640-bbf652ff6d99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:15:52.534234 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.auth.core [None req-bec5c101-38d4-4598-b640-bbf652ff6d99 None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:15:52.536112 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-bec5c101-38d4-4598-b640-bbf652ff6d99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:15:52.543963 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 757/1526] 127.0.0.1 () {58 vars in 998 bytes} [Thu May 19 12:15:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:15:52.698314 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d01fae-c1b8-45c5-b86d-c3e1c2df0089 None None] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:15:52.713498 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d01fae-c1b8-45c5-b86d-c3e1c2df0089 None None] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:15:52.718946 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad30652-069f-41fc-93f2-e65e4b4d975f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34a98d75d84b4c67981377e0f26eca62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd70d25bf69314820b8aaecb65576978a', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:15:52.719618 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad30652-069f-41fc-93f2-e65e4b4d975f service nova] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:15:52.719945 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad30652-069f-41fc-93f2-e65e4b4d975f service nova] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:15:52.720282 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad30652-069f-41fc-93f2-e65e4b4d975f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:15:52.729570 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad30652-069f-41fc-93f2-e65e4b4d975f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:15:52.731203 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad30652-069f-41fc-93f2-e65e4b4d975f service nova] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:15:52.740707 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 770/1527] 10.222.0.23 () {60 vars in 1365 bytes} [Thu May 19 12:15:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:15:56.912941 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba9c4f7-2834-49a1-90b9-cf81d9fcc31a None None] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:15:56.913574 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba9c4f7-2834-49a1-90b9-cf81d9fcc31a None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:15:56.913574 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba9c4f7-2834-49a1-90b9-cf81d9fcc31a None None] PATH_INFO: `/` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:15:56.914843 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 758/1528] 127.0.0.1 () {54 vars in 916 bytes} [Thu May 19 12:15:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 12:15:56.920251 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53406881-605b-4c03-b40d-2f8b8a3cb857 None None] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:15:56.920707 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53406881-605b-4c03-b40d-2f8b8a3cb857 None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:15:56.920835 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53406881-605b-4c03-b40d-2f8b8a3cb857 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:15:56.939633 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.auth.core [None req-53406881-605b-4c03-b40d-2f8b8a3cb857 None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:15:56.940861 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-53406881-605b-4c03-b40d-2f8b8a3cb857 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:15:56.949938 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 771/1529] 127.0.0.1 () {58 vars in 1013 bytes} [Thu May 19 12:15:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 12:15:56.986627 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba9c4f7-2834-49a1-90b9-cf81d9fcc31a None None] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:15:56.989237 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-6ba9c4f7-2834-49a1-90b9-cf81d9fcc31a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:15:57.046531 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba9c4f7-2834-49a1-90b9-cf81d9fcc31a None None] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:15:57.050051 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d76842b-ed76-4312-9806-d84140e54cbf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a00b3fa7497456895e5e0bf105c4fc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd70d25bf69314820b8aaecb65576978a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:15:57.051181 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d76842b-ed76-4312-9806-d84140e54cbf service neutron] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:15:57.051632 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d76842b-ed76-4312-9806-d84140e54cbf service neutron] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:15:57.051898 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d76842b-ed76-4312-9806-d84140e54cbf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:15:57.060944 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d76842b-ed76-4312-9806-d84140e54cbf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:15:57.062409 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d76842b-ed76-4312-9806-d84140e54cbf service neutron] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:15:57.072771 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 759/1530] 10.222.0.23 () {60 vars in 1365 bytes} [Thu May 19 12:15:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:15:59.345876 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49ac944-16ff-48cc-ad66-099961d1b288 None None] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:15:59.346690 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49ac944-16ff-48cc-ad66-099961d1b288 None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:15:59.346690 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49ac944-16ff-48cc-ad66-099961d1b288 None None] PATH_INFO: `/` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:15:59.348140 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 772/1531] 127.0.0.1 () {54 vars in 916 bytes} [Thu May 19 12:15:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 12:15:59.353655 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98721d46-8c4f-4c46-a472-9d5bc1e78318 None None] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:15:59.354132 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98721d46-8c4f-4c46-a472-9d5bc1e78318 None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:15:59.354238 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98721d46-8c4f-4c46-a472-9d5bc1e78318 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:15:59.373238 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.auth.core [None req-98721d46-8c4f-4c46-a472-9d5bc1e78318 None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:15:59.374460 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-98721d46-8c4f-4c46-a472-9d5bc1e78318 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:15:59.382957 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 760/1532] 127.0.0.1 () {58 vars in 1013 bytes} [Thu May 19 12:15:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:15:59.418145 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49ac944-16ff-48cc-ad66-099961d1b288 None None] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:15:59.429884 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49ac944-16ff-48cc-ad66-099961d1b288 None None] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:15:59.433584 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d606240-ad1c-464f-9a4c-af7b5cf0aea3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a00b3fa7497456895e5e0bf105c4fc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd70d25bf69314820b8aaecb65576978a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:15:59.434230 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d606240-ad1c-464f-9a4c-af7b5cf0aea3 service neutron] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:15:59.434522 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d606240-ad1c-464f-9a4c-af7b5cf0aea3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:15:59.434802 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d606240-ad1c-464f-9a4c-af7b5cf0aea3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:15:59.444822 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d606240-ad1c-464f-9a4c-af7b5cf0aea3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:15:59.446266 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d606240-ad1c-464f-9a4c-af7b5cf0aea3 service neutron] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:15:59.455756 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 773/1533] 10.222.0.23 () {60 vars in 1365 bytes} [Thu May 19 12:15:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:16:03.852483 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88131153-d75d-4a67-a48f-03e3206f8835 None None] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:16:03.853050 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88131153-d75d-4a67-a48f-03e3206f8835 None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:16:03.853050 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88131153-d75d-4a67-a48f-03e3206f8835 None None] PATH_INFO: `/` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:16:03.854387 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 761/1534] 127.0.0.1 () {54 vars in 916 bytes} [Thu May 19 12:16:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 12:16:03.859586 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220962eb-66e9-42d0-ae4e-f0c36b4edb92 None None] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:16:03.860084 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220962eb-66e9-42d0-ae4e-f0c36b4edb92 None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:16:03.860191 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220962eb-66e9-42d0-ae4e-f0c36b4edb92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:16:03.878703 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.auth.core [None req-220962eb-66e9-42d0-ae4e-f0c36b4edb92 None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:16:03.880798 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-220962eb-66e9-42d0-ae4e-f0c36b4edb92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:16:03.888631 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 774/1535] 127.0.0.1 () {58 vars in 1013 bytes} [Thu May 19 12:16:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:16:04.165205 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88131153-d75d-4a67-a48f-03e3206f8835 None None] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:16:04.166963 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-88131153-d75d-4a67-a48f-03e3206f8835 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:16:04.172872 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=85 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:16:04.227287 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88131153-d75d-4a67-a48f-03e3206f8835 None None] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:16:04.230819 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d873fe-ab39-4fe3-844a-7ac33cdeffe1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8800254c0e784e42a791fb9abc274d79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd70d25bf69314820b8aaecb65576978a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:16:04.231458 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d873fe-ab39-4fe3-844a-7ac33cdeffe1 service cinder] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:16:04.231767 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d873fe-ab39-4fe3-844a-7ac33cdeffe1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:16:04.232086 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d873fe-ab39-4fe3-844a-7ac33cdeffe1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:16:04.242137 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d873fe-ab39-4fe3-844a-7ac33cdeffe1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:16:04.243753 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d873fe-ab39-4fe3-844a-7ac33cdeffe1 service cinder] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:16:04.253392 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 762/1536] 10.222.0.23 () {60 vars in 1365 bytes} [Thu May 19 12:16:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:16:06.571569 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32de27e2-14fb-4e83-ad6b-5d3d8f07a314 None None] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:16:06.572323 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32de27e2-14fb-4e83-ad6b-5d3d8f07a314 None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:16:06.572422 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32de27e2-14fb-4e83-ad6b-5d3d8f07a314 None None] PATH_INFO: `/` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:16:06.573865 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 775/1537] 127.0.0.1 () {54 vars in 916 bytes} [Thu May 19 12:16:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 12:16:06.580125 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aae67c9-cba0-47cd-b114-b81b0f917afd None None] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:16:06.580681 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aae67c9-cba0-47cd-b114-b81b0f917afd None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:16:06.580681 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aae67c9-cba0-47cd-b114-b81b0f917afd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:16:06.602090 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.auth.core [None req-7aae67c9-cba0-47cd-b114-b81b0f917afd None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:16:06.603342 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-7aae67c9-cba0-47cd-b114-b81b0f917afd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:16:06.613412 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 763/1538] 127.0.0.1 () {58 vars in 1013 bytes} [Thu May 19 12:16:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 12:16:06.898068 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32de27e2-14fb-4e83-ad6b-5d3d8f07a314 None None] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:16:06.900235 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-32de27e2-14fb-4e83-ad6b-5d3d8f07a314 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:16:06.908348 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=84 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:16:06.915472 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32de27e2-14fb-4e83-ad6b-5d3d8f07a314 None None] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:16:06.927677 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f17fa82-3c61-4a33-bea0-6cf84f182960 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8800254c0e784e42a791fb9abc274d79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd70d25bf69314820b8aaecb65576978a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:16:06.927677 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f17fa82-3c61-4a33-bea0-6cf84f182960 service cinder] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:16:06.927677 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f17fa82-3c61-4a33-bea0-6cf84f182960 service cinder] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:16:06.927677 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f17fa82-3c61-4a33-bea0-6cf84f182960 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:16:06.930966 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f17fa82-3c61-4a33-bea0-6cf84f182960 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:16:06.932422 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f17fa82-3c61-4a33-bea0-6cf84f182960 service cinder] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:16:06.943427 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 776/1539] 10.222.0.23 () {60 vars in 1365 bytes} [Thu May 19 12:16:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:16:16.595805 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=28 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:16:16.960812 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=5 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:17:43.987003 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b74f39-70c2-49e5-80cf-ac525d4bf3ba None None] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:17:43.987777 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b74f39-70c2-49e5-80cf-ac525d4bf3ba None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:17:43.987910 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52644db0-30c6-4112-84fe-3c45f51ebee3 None None] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:17:43.988309 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b74f39-70c2-49e5-80cf-ac525d4bf3ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:17:43.988412 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52644db0-30c6-4112-84fe-3c45f51ebee3 None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:17:43.988412 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52644db0-30c6-4112-84fe-3c45f51ebee3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:17:44.024650 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.auth.core [None req-34b74f39-70c2-49e5-80cf-ac525d4bf3ba None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:17:44.026147 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-34b74f39-70c2-49e5-80cf-ac525d4bf3ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:17:44.026812 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.auth.core [None req-52644db0-30c6-4112-84fe-3c45f51ebee3 None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:17:44.028016 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-52644db0-30c6-4112-84fe-3c45f51ebee3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:17:44.041329 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 764/1540] 10.222.0.23 () {58 vars in 957 bytes} [Thu May 19 12:17:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:17:44.050512 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 777/1541] 10.222.0.23 () {58 vars in 957 bytes} [Thu May 19 12:17:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 12:17:44.068334 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1776190-fbd2-418e-9dc5-bd04ec3b35b7 None None] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:17:44.069332 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1776190-fbd2-418e-9dc5-bd04ec3b35b7 None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:17:44.070619 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1776190-fbd2-418e-9dc5-bd04ec3b35b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:17:44.073563 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3a0090-8229-460f-8bb3-db5457412ade None None] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:17:44.080035 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3a0090-8229-460f-8bb3-db5457412ade None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:17:44.080035 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3a0090-8229-460f-8bb3-db5457412ade None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:17:44.097929 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.auth.core [None req-c1776190-fbd2-418e-9dc5-bd04ec3b35b7 None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:17:44.099335 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-c1776190-fbd2-418e-9dc5-bd04ec3b35b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:17:44.112575 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 765/1542] 10.222.0.23 () {58 vars in 957 bytes} [Thu May 19 12:17:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:17:44.119347 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.auth.core [None req-4c3a0090-8229-460f-8bb3-db5457412ade None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:17:44.120709 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-4c3a0090-8229-460f-8bb3-db5457412ade None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:17:44.122982 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1776190-fbd2-418e-9dc5-bd04ec3b35b7 None None] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:17:44.140847 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1776190-fbd2-418e-9dc5-bd04ec3b35b7 None None] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:17:44.143891 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 778/1543] 10.222.0.23 () {58 vars in 957 bytes} [Thu May 19 12:17:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:17:44.146154 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f381fe51-8b80-4e59-8cf4-eec76e2f24b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:17:44.147108 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f381fe51-8b80-4e59-8cf4-eec76e2f24b3 admin admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:17:44.147400 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f381fe51-8b80-4e59-8cf4-eec76e2f24b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:17:44.147772 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f381fe51-8b80-4e59-8cf4-eec76e2f24b3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:17:44.149053 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3a0090-8229-460f-8bb3-db5457412ade None None] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:17:44.152807 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f381fe51-8b80-4e59-8cf4-eec76e2f24b3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:17:44.153225 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f381fe51-8b80-4e59-8cf4-eec76e2f24b3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:17:44.154236 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: WARNING py.warnings [None req-f381fe51-8b80-4e59-8cf4-eec76e2f24b3 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 19 12:17:44.154236 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: warnings.warn(msg) May 19 12:17:44.154236 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: May 19 12:17:44.155167 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f381fe51-8b80-4e59-8cf4-eec76e2f24b3 admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:17:44.170387 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3a0090-8229-460f-8bb3-db5457412ade None None] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:17:44.183987 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 766/1544] 10.222.0.23 () {58 vars in 1152 bytes} [Thu May 19 12:17:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 12:17:44.191206 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6beb27-1668-498a-a17b-c8ca1c1b9687 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:17:44.191937 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6beb27-1668-498a-a17b-c8ca1c1b9687 admin admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:17:44.192229 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6beb27-1668-498a-a17b-c8ca1c1b9687 admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:17:44.192508 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6beb27-1668-498a-a17b-c8ca1c1b9687 admin admin] PATH_INFO: `/v3/domains` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:17:44.193256 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6beb27-1668-498a-a17b-c8ca1c1b9687 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:17:44.193571 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6beb27-1668-498a-a17b-c8ca1c1b9687 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:17:44.195278 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6beb27-1668-498a-a17b-c8ca1c1b9687 admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:17:44.207696 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f381fe51-8b80-4e59-8cf4-eec76e2f24b3 admin admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:17:44.224117 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 779/1545] 10.222.0.23 () {58 vars in 1152 bytes} [Thu May 19 12:17:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:17:44.231814 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6beb27-1668-498a-a17b-c8ca1c1b9687 admin admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:17:44.238509 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f381fe51-8b80-4e59-8cf4-eec76e2f24b3 admin admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:17:44.244152 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7479c3f-3ba5-4456-a04f-389396038551 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:17:44.244834 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7479c3f-3ba5-4456-a04f-389396038551 admin admin] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:17:44.245114 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7479c3f-3ba5-4456-a04f-389396038551 admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:17:44.245390 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7479c3f-3ba5-4456-a04f-389396038551 admin admin] PATH_INFO: `/v3/projects` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:17:44.246175 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6beb27-1668-498a-a17b-c8ca1c1b9687 admin admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:17:44.246306 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7479c3f-3ba5-4456-a04f-389396038551 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:17:44.252720 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7479c3f-3ba5-4456-a04f-389396038551 admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:17:44.263619 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b56679-25e7-46ad-8e9a-e954785ecc70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:17:44.264545 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b56679-25e7-46ad-8e9a-e954785ecc70 admin admin] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:17:44.264839 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b56679-25e7-46ad-8e9a-e954785ecc70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:17:44.265118 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b56679-25e7-46ad-8e9a-e954785ecc70 admin admin] PATH_INFO: `/v3/projects` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:17:44.265851 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b56679-25e7-46ad-8e9a-e954785ecc70 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:17:44.267492 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: WARNING py.warnings [None req-13b56679-25e7-46ad-8e9a-e954785ecc70 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 19 12:17:44.267492 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: warnings.warn(msg) May 19 12:17:44.267492 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: May 19 12:17:44.275711 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b56679-25e7-46ad-8e9a-e954785ecc70 admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:17:44.363137 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 767/1546] 10.222.0.23 () {60 vars in 1152 bytes} [Thu May 19 12:17:44 2022] POST /identity/v3/projects => generated 387 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 12:17:44.368484 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7479c3f-3ba5-4456-a04f-389396038551 admin admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:17:44.394971 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 780/1547] 10.222.0.23 () {60 vars in 1152 bytes} [Thu May 19 12:17:44 2022] POST /identity/v3/projects => generated 393 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 12:17:44.403210 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b56679-25e7-46ad-8e9a-e954785ecc70 admin admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:17:44.483277 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7479c3f-3ba5-4456-a04f-389396038551 admin admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:17:44.518393 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b56679-25e7-46ad-8e9a-e954785ecc70 admin admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:17:44.528368 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e42e2b-2d3f-4926-87f4-9dea3fda7582 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:17:44.529078 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e42e2b-2d3f-4926-87f4-9dea3fda7582 admin admin] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:17:44.529353 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e42e2b-2d3f-4926-87f4-9dea3fda7582 admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:17:44.529634 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e42e2b-2d3f-4926-87f4-9dea3fda7582 admin admin] PATH_INFO: `/v3/users` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:17:44.530346 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e42e2b-2d3f-4926-87f4-9dea3fda7582 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:17:44.532373 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e42e2b-2d3f-4926-87f4-9dea3fda7582 admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:17:44.624511 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 781/1548] 10.222.0.23 () {60 vars in 1143 bytes} [Thu May 19 12:17:44 2022] POST /identity/v3/users => generated 297 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 12:17:44.625936 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6b2a65-db23-4641-ad2d-1841349e641b None None] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:17:44.626236 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6b2a65-db23-4641-ad2d-1841349e641b None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:17:44.626524 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6b2a65-db23-4641-ad2d-1841349e641b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:17:44.631562 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dab141a-542e-498f-bc69-5d81eb2e7a6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:17:44.639853 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dab141a-542e-498f-bc69-5d81eb2e7a6f admin admin] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:17:44.639853 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dab141a-542e-498f-bc69-5d81eb2e7a6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:17:44.639853 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dab141a-542e-498f-bc69-5d81eb2e7a6f admin admin] PATH_INFO: `/v3/users` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:17:44.639853 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dab141a-542e-498f-bc69-5d81eb2e7a6f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:17:44.639853 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: WARNING py.warnings [None req-9dab141a-542e-498f-bc69-5d81eb2e7a6f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 19 12:17:44.639853 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: warnings.warn(msg) May 19 12:17:44.639853 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: May 19 12:17:44.641072 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dab141a-542e-498f-bc69-5d81eb2e7a6f admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:17:44.665450 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.auth.core [None req-0e6b2a65-db23-4641-ad2d-1841349e641b None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:17:44.667024 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-0e6b2a65-db23-4641-ad2d-1841349e641b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:17:44.677874 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 782/1549] 10.222.0.23 () {58 vars in 957 bytes} [Thu May 19 12:17:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:17:44.678825 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6b2a65-db23-4641-ad2d-1841349e641b None None] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:17:44.690638 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6b2a65-db23-4641-ad2d-1841349e641b None None] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:17:44.700858 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9754d58-42f6-42e6-984c-75f2a99de583 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:17:44.702932 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9754d58-42f6-42e6-984c-75f2a99de583 admin admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:17:44.702932 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9754d58-42f6-42e6-984c-75f2a99de583 admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:17:44.702932 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9754d58-42f6-42e6-984c-75f2a99de583 admin admin] PATH_INFO: `/v3/roles` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:17:44.702932 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9754d58-42f6-42e6-984c-75f2a99de583 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:17:44.704823 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9754d58-42f6-42e6-984c-75f2a99de583 admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:17:44.720237 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 783/1550] 10.222.0.23 () {58 vars in 1121 bytes} [Thu May 19 12:17:44 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 12:17:44.721882 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9001dfe5-e836-4f0d-b844-037e0892f9ec None None] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:17:44.722168 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9001dfe5-e836-4f0d-b844-037e0892f9ec None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:17:44.722437 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9001dfe5-e836-4f0d-b844-037e0892f9ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:17:44.730875 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 768/1551] 10.222.0.23 () {60 vars in 1143 bytes} [Thu May 19 12:17:44 2022] POST /identity/v3/users => generated 294 bytes in 364 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 12:17:44.732406 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef34dd2-b0ae-4b24-925c-912e4d430602 None None] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:17:44.732742 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef34dd2-b0ae-4b24-925c-912e4d430602 None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:17:44.733074 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef34dd2-b0ae-4b24-925c-912e4d430602 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:17:44.754544 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.auth.core [None req-9ef34dd2-b0ae-4b24-925c-912e4d430602 None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:17:44.755999 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-9ef34dd2-b0ae-4b24-925c-912e4d430602 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:17:44.760381 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.auth.core [None req-9001dfe5-e836-4f0d-b844-037e0892f9ec None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:17:44.762780 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-9001dfe5-e836-4f0d-b844-037e0892f9ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:17:44.769949 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 769/1552] 10.222.0.23 () {58 vars in 957 bytes} [Thu May 19 12:17:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:17:44.774462 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbce4eae-7295-4ea4-913c-6fa74e0c51ca None None] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:17:44.774826 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbce4eae-7295-4ea4-913c-6fa74e0c51ca None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:17:44.776425 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbce4eae-7295-4ea4-913c-6fa74e0c51ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:17:44.782933 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 784/1553] 10.222.0.23 () {58 vars in 957 bytes} [Thu May 19 12:17:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:17:44.783845 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9001dfe5-e836-4f0d-b844-037e0892f9ec None None] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:17:44.813937 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9001dfe5-e836-4f0d-b844-037e0892f9ec None None] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:17:44.819230 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.auth.core [None req-bbce4eae-7295-4ea4-913c-6fa74e0c51ca None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:17:44.833243 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-bbce4eae-7295-4ea4-913c-6fa74e0c51ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:17:44.840784 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9e2a7b-bdb0-4732-96da-df8e95398903 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:17:44.841476 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9e2a7b-bdb0-4732-96da-df8e95398903 admin admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:17:44.841753 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9e2a7b-bdb0-4732-96da-df8e95398903 admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:17:44.842033 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9e2a7b-bdb0-4732-96da-df8e95398903 admin admin] PATH_INFO: `/v3/roles` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:17:44.843360 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9e2a7b-bdb0-4732-96da-df8e95398903 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:17:44.845069 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9e2a7b-bdb0-4732-96da-df8e95398903 admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:17:44.850212 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 770/1554] 10.222.0.23 () {58 vars in 957 bytes} [Thu May 19 12:17:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:17:44.853311 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbce4eae-7295-4ea4-913c-6fa74e0c51ca None None] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:17:44.858108 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 785/1555] 10.222.0.23 () {58 vars in 1121 bytes} [Thu May 19 12:17:44 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 12:17:44.859254 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9e2a7b-bdb0-4732-96da-df8e95398903 admin admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:17:44.865892 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbce4eae-7295-4ea4-913c-6fa74e0c51ca None None] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:17:44.871565 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9e2a7b-bdb0-4732-96da-df8e95398903 admin admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:17:44.873867 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bca2dd-13dd-4509-8579-1267df39183f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:17:44.875064 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bca2dd-13dd-4509-8579-1267df39183f admin admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:17:44.875355 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bca2dd-13dd-4509-8579-1267df39183f admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:17:44.875686 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bca2dd-13dd-4509-8579-1267df39183f admin admin] PATH_INFO: `/v3/roles` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:17:44.876886 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26bca2dd-13dd-4509-8579-1267df39183f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:17:44.878792 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26bca2dd-13dd-4509-8579-1267df39183f admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:17:44.879142 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380ba967-6995-4329-8e70-4eb6ad54c024 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:17:44.880124 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380ba967-6995-4329-8e70-4eb6ad54c024 admin admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:17:44.880671 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380ba967-6995-4329-8e70-4eb6ad54c024 admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:17:44.881256 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380ba967-6995-4329-8e70-4eb6ad54c024 admin admin] PATH_INFO: `/v3/domains` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:17:44.882860 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380ba967-6995-4329-8e70-4eb6ad54c024 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:17:44.883551 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380ba967-6995-4329-8e70-4eb6ad54c024 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:17:44.885777 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380ba967-6995-4329-8e70-4eb6ad54c024 admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:17:44.893405 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 771/1556] 10.222.0.23 () {58 vars in 1121 bytes} [Thu May 19 12:17:44 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 12:17:44.898842 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744509e1-3463-4d26-869b-53acac6ddc2d None None] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:17:44.898842 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744509e1-3463-4d26-869b-53acac6ddc2d None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:17:44.898842 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744509e1-3463-4d26-869b-53acac6ddc2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:17:44.907999 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 786/1557] 10.222.0.23 () {58 vars in 1152 bytes} [Thu May 19 12:17:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:17:44.907999 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db6332d-f9dc-46e8-957b-d1e9ba52db34 None None] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:17:44.907999 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db6332d-f9dc-46e8-957b-d1e9ba52db34 None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:17:44.907999 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db6332d-f9dc-46e8-957b-d1e9ba52db34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:17:44.932869 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.auth.core [None req-744509e1-3463-4d26-869b-53acac6ddc2d None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:17:44.934835 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-744509e1-3463-4d26-869b-53acac6ddc2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:17:44.938932 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.auth.core [None req-4db6332d-f9dc-46e8-957b-d1e9ba52db34 None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:17:44.940403 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-4db6332d-f9dc-46e8-957b-d1e9ba52db34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:17:44.947147 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 772/1558] 10.222.0.23 () {58 vars in 957 bytes} [Thu May 19 12:17:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:17:44.950534 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744509e1-3463-4d26-869b-53acac6ddc2d None None] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:17:44.969974 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744509e1-3463-4d26-869b-53acac6ddc2d None None] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:17:44.973427 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 787/1559] 10.222.0.23 () {58 vars in 957 bytes} [Thu May 19 12:17:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 68 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:17:44.976700 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566740a6-30ba-4dda-aa9c-88d5c6f4827f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:17:44.977321 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf305f97-9204-4ddc-8b32-7768e5bdfba0 None None] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:17:44.977920 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf305f97-9204-4ddc-8b32-7768e5bdfba0 None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:17:44.978490 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf305f97-9204-4ddc-8b32-7768e5bdfba0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:17:44.979040 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566740a6-30ba-4dda-aa9c-88d5c6f4827f admin admin] REQUEST_METHOD: `PUT` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:17:44.979686 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566740a6-30ba-4dda-aa9c-88d5c6f4827f admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:17:44.980200 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566740a6-30ba-4dda-aa9c-88d5c6f4827f admin admin] PATH_INFO: `/v3/projects/b3d651a605fd4dee95797a7ec3896792/users/8e445569dc5641ed962b0ac977850a6c/roles/259eb400cdb4439db72d14456ae9cd08` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:17:45.001998 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566740a6-30ba-4dda-aa9c-88d5c6f4827f admin admin] RBAC: Authorizing `identity:create_grant(project_id=b3d651a605fd4dee95797a7ec3896792, user_id=8e445569dc5641ed962b0ac977850a6c, role_id=259eb400cdb4439db72d14456ae9cd08)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:17:45.006294 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566740a6-30ba-4dda-aa9c-88d5c6f4827f admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:17:45.010260 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:17:45.013437 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.auth.core [None req-cf305f97-9204-4ddc-8b32-7768e5bdfba0 None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:17:45.014735 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-cf305f97-9204-4ddc-8b32-7768e5bdfba0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:17:45.024161 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 788/1560] 10.222.0.23 () {58 vars in 957 bytes} [Thu May 19 12:17:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:17:45.024839 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf305f97-9204-4ddc-8b32-7768e5bdfba0 None None] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:17:45.039074 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf305f97-9204-4ddc-8b32-7768e5bdfba0 None None] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:17:45.052519 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84fd647-f40f-4122-813b-86a5592c4146 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:17:45.053703 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84fd647-f40f-4122-813b-86a5592c4146 admin admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:17:45.054782 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84fd647-f40f-4122-813b-86a5592c4146 admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:17:45.055292 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84fd647-f40f-4122-813b-86a5592c4146 admin admin] PATH_INFO: `/v3/roles` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:17:45.056845 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84fd647-f40f-4122-813b-86a5592c4146 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:17:45.060047 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84fd647-f40f-4122-813b-86a5592c4146 admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:17:45.080313 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 789/1561] 10.222.0.23 () {58 vars in 1121 bytes} [Thu May 19 12:17:45 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 12:17:45.080313 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84fd647-f40f-4122-813b-86a5592c4146 admin admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:17:45.094001 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84fd647-f40f-4122-813b-86a5592c4146 admin admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:17:45.099577 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f5ddd4-890b-45b1-b77f-2c2a9a98a1d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:17:45.099577 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f5ddd4-890b-45b1-b77f-2c2a9a98a1d4 admin admin] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:17:45.100360 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f5ddd4-890b-45b1-b77f-2c2a9a98a1d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:17:45.100360 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f5ddd4-890b-45b1-b77f-2c2a9a98a1d4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:17:45.101577 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f5ddd4-890b-45b1-b77f-2c2a9a98a1d4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:17:45.103636 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f5ddd4-890b-45b1-b77f-2c2a9a98a1d4 admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:17:45.125572 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 773/1562] 10.222.0.23 () {60 vars in 1482 bytes} [Thu May 19 12:17:44 2022] PUT /identity/v3/projects/b3d651a605fd4dee95797a7ec3896792/users/8e445569dc5641ed962b0ac977850a6c/roles/259eb400cdb4439db72d14456ae9cd08 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:17:45.127377 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566740a6-30ba-4dda-aa9c-88d5c6f4827f admin admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:17:45.195921 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566740a6-30ba-4dda-aa9c-88d5c6f4827f admin admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:17:45.201019 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9972f71e-dea7-440d-81a8-08649c7ce203 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:17:45.201729 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9972f71e-dea7-440d-81a8-08649c7ce203 admin admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:17:45.202014 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9972f71e-dea7-440d-81a8-08649c7ce203 admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:17:45.202316 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9972f71e-dea7-440d-81a8-08649c7ce203 admin admin] PATH_INFO: `/v3/domains` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:17:45.203068 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9972f71e-dea7-440d-81a8-08649c7ce203 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:17:45.203380 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9972f71e-dea7-440d-81a8-08649c7ce203 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:17:45.205216 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9972f71e-dea7-440d-81a8-08649c7ce203 admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:17:45.206902 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 790/1563] 10.222.0.23 () {60 vars in 1152 bytes} [Thu May 19 12:17:45 2022] POST /identity/v3/projects => generated 379 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 12:17:45.208365 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b50225-5e2f-49ab-8f5a-729e4170099d None None] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:17:45.212969 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b50225-5e2f-49ab-8f5a-729e4170099d None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:17:45.212969 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b50225-5e2f-49ab-8f5a-729e4170099d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:17:45.221810 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 774/1564] 10.222.0.23 () {58 vars in 1152 bytes} [Thu May 19 12:17:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:17:45.222770 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9972f71e-dea7-440d-81a8-08649c7ce203 admin admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:17:45.223926 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:17:45.239202 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.auth.core [None req-29b50225-5e2f-49ab-8f5a-729e4170099d None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:17:45.241111 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-29b50225-5e2f-49ab-8f5a-729e4170099d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:17:45.247013 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9972f71e-dea7-440d-81a8-08649c7ce203 admin admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:17:45.310601 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589736db-bc70-4efe-8807-cede767814ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:17:45.311285 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589736db-bc70-4efe-8807-cede767814ca admin admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:17:45.311576 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589736db-bc70-4efe-8807-cede767814ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:17:45.311882 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589736db-bc70-4efe-8807-cede767814ca admin admin] PATH_INFO: `/v3/domains` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:17:45.312628 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589736db-bc70-4efe-8807-cede767814ca admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:17:45.312939 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589736db-bc70-4efe-8807-cede767814ca admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:17:45.314557 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589736db-bc70-4efe-8807-cede767814ca admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:17:45.318812 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 791/1565] 10.222.0.23 () {58 vars in 957 bytes} [Thu May 19 12:17:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:17:45.320973 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2cd0ae-be3b-491d-917c-b29c9b8b053c None None] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:17:45.321321 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2cd0ae-be3b-491d-917c-b29c9b8b053c None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:17:45.321637 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2cd0ae-be3b-491d-917c-b29c9b8b053c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:17:45.336994 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 775/1566] 10.222.0.23 () {58 vars in 1152 bytes} [Thu May 19 12:17:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:17:45.336994 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589736db-bc70-4efe-8807-cede767814ca admin admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:17:45.342648 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589736db-bc70-4efe-8807-cede767814ca admin admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:17:45.345106 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.auth.core [None req-9c2cd0ae-be3b-491d-917c-b29c9b8b053c None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:17:45.346334 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-9c2cd0ae-be3b-491d-917c-b29c9b8b053c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:17:45.347493 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa0e145-ad7d-49db-beec-0df768cbe47a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:17:45.348487 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa0e145-ad7d-49db-beec-0df768cbe47a admin admin] REQUEST_METHOD: `PUT` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:17:45.348778 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa0e145-ad7d-49db-beec-0df768cbe47a admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:17:45.349049 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa0e145-ad7d-49db-beec-0df768cbe47a admin admin] PATH_INFO: `/v3/projects/482e9ea9fe48482fa644191c2956b4ec/users/d95e4c71778448eb96109cf89365522a/roles/259eb400cdb4439db72d14456ae9cd08` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:17:45.354837 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 792/1567] 10.222.0.23 () {58 vars in 957 bytes} [Thu May 19 12:17:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:17:45.356356 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd01d82-da52-4589-9ddb-2c8b8362c805 None None] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:17:45.356653 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd01d82-da52-4589-9ddb-2c8b8362c805 None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:17:45.356928 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd01d82-da52-4589-9ddb-2c8b8362c805 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:17:45.371478 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa0e145-ad7d-49db-beec-0df768cbe47a admin admin] RBAC: Authorizing `identity:create_grant(project_id=482e9ea9fe48482fa644191c2956b4ec, user_id=d95e4c71778448eb96109cf89365522a, role_id=259eb400cdb4439db72d14456ae9cd08)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:17:45.374192 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa0e145-ad7d-49db-beec-0df768cbe47a admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:17:45.378560 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.auth.core [None req-7cd01d82-da52-4589-9ddb-2c8b8362c805 None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:17:45.379858 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-7cd01d82-da52-4589-9ddb-2c8b8362c805 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:17:45.392263 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 793/1568] 10.222.0.23 () {58 vars in 957 bytes} [Thu May 19 12:17:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:17:45.394452 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c7b0be-d462-45f7-8e4f-9620ebea23bd None None] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:17:45.394911 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c7b0be-d462-45f7-8e4f-9620ebea23bd None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:17:45.395331 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c7b0be-d462-45f7-8e4f-9620ebea23bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:17:45.414428 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.auth.core [None req-a5c7b0be-d462-45f7-8e4f-9620ebea23bd None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:17:45.424990 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-a5c7b0be-d462-45f7-8e4f-9620ebea23bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:17:45.424990 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 794/1569] 10.222.0.23 () {58 vars in 957 bytes} [Thu May 19 12:17:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:17:45.424990 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c7b0be-d462-45f7-8e4f-9620ebea23bd None None] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:17:45.435671 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c7b0be-d462-45f7-8e4f-9620ebea23bd None None] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:17:45.440546 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f71b0b-200d-41c5-9857-bd5d4a51a3ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:17:45.441254 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f71b0b-200d-41c5-9857-bd5d4a51a3ad admin admin] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:17:45.441540 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f71b0b-200d-41c5-9857-bd5d4a51a3ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:17:45.441540 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f71b0b-200d-41c5-9857-bd5d4a51a3ad admin admin] PATH_INFO: `/v3/users` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:17:45.442514 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f71b0b-200d-41c5-9857-bd5d4a51a3ad admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:17:45.446916 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f71b0b-200d-41c5-9857-bd5d4a51a3ad admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:17:45.455196 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 776/1570] 10.222.0.23 () {60 vars in 1482 bytes} [Thu May 19 12:17:45 2022] PUT /identity/v3/projects/482e9ea9fe48482fa644191c2956b4ec/users/d95e4c71778448eb96109cf89365522a/roles/259eb400cdb4439db72d14456ae9cd08 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:17:45.456063 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa0e145-ad7d-49db-beec-0df768cbe47a admin admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:17:45.523658 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:17:45.527369 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa0e145-ad7d-49db-beec-0df768cbe47a admin admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:17:45.532102 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b1b2fd-3310-4a65-82b5-21a94fbd3434 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:17:45.532769 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b1b2fd-3310-4a65-82b5-21a94fbd3434 admin admin] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:17:45.533056 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b1b2fd-3310-4a65-82b5-21a94fbd3434 admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:17:45.533346 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b1b2fd-3310-4a65-82b5-21a94fbd3434 admin admin] PATH_INFO: `/v3/projects` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:17:45.534045 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b1b2fd-3310-4a65-82b5-21a94fbd3434 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:17:45.536002 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b1b2fd-3310-4a65-82b5-21a94fbd3434 admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:17:45.563777 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 795/1571] 10.222.0.23 () {60 vars in 1143 bytes} [Thu May 19 12:17:45 2022] POST /identity/v3/users => generated 290 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 12:17:45.564066 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f71b0b-200d-41c5-9857-bd5d4a51a3ad admin admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:17:45.574652 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f71b0b-200d-41c5-9857-bd5d4a51a3ad admin admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:17:45.579509 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e46f73f-5399-456b-b57e-05d5efc97a61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:17:45.580254 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e46f73f-5399-456b-b57e-05d5efc97a61 admin admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:17:45.580546 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e46f73f-5399-456b-b57e-05d5efc97a61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:17:45.580832 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e46f73f-5399-456b-b57e-05d5efc97a61 admin admin] PATH_INFO: `/v3/domains` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:17:45.582176 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e46f73f-5399-456b-b57e-05d5efc97a61 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:17:45.582176 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e46f73f-5399-456b-b57e-05d5efc97a61 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:17:45.583509 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e46f73f-5399-456b-b57e-05d5efc97a61 admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:17:45.597179 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 796/1572] 10.222.0.23 () {58 vars in 1152 bytes} [Thu May 19 12:17:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:17:45.597995 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e46f73f-5399-456b-b57e-05d5efc97a61 admin admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:17:45.608481 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e46f73f-5399-456b-b57e-05d5efc97a61 admin admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:17:45.613365 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c4f756-d8dc-4978-8b95-90dd3384ec8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:17:45.621753 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c4f756-d8dc-4978-8b95-90dd3384ec8f admin admin] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:17:45.621753 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c4f756-d8dc-4978-8b95-90dd3384ec8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:17:45.621753 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c4f756-d8dc-4978-8b95-90dd3384ec8f admin admin] PATH_INFO: `/v3/projects` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:17:45.621753 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c4f756-d8dc-4978-8b95-90dd3384ec8f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:17:45.621753 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c4f756-d8dc-4978-8b95-90dd3384ec8f admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:17:45.622173 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 777/1573] 10.222.0.23 () {60 vars in 1152 bytes} [Thu May 19 12:17:45 2022] POST /identity/v3/projects => generated 375 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 12:17:45.622703 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a80e31d-4d15-4e2d-9510-d2808e34c6a5 None None] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:17:45.622991 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a80e31d-4d15-4e2d-9510-d2808e34c6a5 None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:17:45.623278 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a80e31d-4d15-4e2d-9510-d2808e34c6a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:17:45.644236 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.auth.core [None req-9a80e31d-4d15-4e2d-9510-d2808e34c6a5 None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:17:45.645445 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-9a80e31d-4d15-4e2d-9510-d2808e34c6a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:17:45.707948 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 797/1574] 10.222.0.23 () {60 vars in 1152 bytes} [Thu May 19 12:17:45 2022] POST /identity/v3/projects => generated 381 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 12:17:45.707948 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9f3f80-d5dc-47ed-8d09-c154fea99059 None None] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:17:45.707948 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9f3f80-d5dc-47ed-8d09-c154fea99059 None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:17:45.707948 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9f3f80-d5dc-47ed-8d09-c154fea99059 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:17:45.716682 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 778/1575] 10.222.0.23 () {58 vars in 957 bytes} [Thu May 19 12:17:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:17:45.717656 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a80e31d-4d15-4e2d-9510-d2808e34c6a5 None None] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:17:45.728070 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.auth.core [None req-dc9f3f80-d5dc-47ed-8d09-c154fea99059 None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:17:45.729515 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-dc9f3f80-d5dc-47ed-8d09-c154fea99059 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:17:45.787803 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a80e31d-4d15-4e2d-9510-d2808e34c6a5 None None] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:17:45.804938 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 798/1576] 10.222.0.23 () {58 vars in 957 bytes} [Thu May 19 12:17:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:17:45.806356 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316814f1-7d35-4ae4-9298-0b73d9f66a69 None None] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:17:45.806649 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316814f1-7d35-4ae4-9298-0b73d9f66a69 None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:17:45.806935 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316814f1-7d35-4ae4-9298-0b73d9f66a69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:17:45.837419 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.auth.core [None req-316814f1-7d35-4ae4-9298-0b73d9f66a69 None None] MFA Rules not processed for user `d95e4c71778448eb96109cf89365522a`. Rule list: `[]` (Enabled: `True`). {{(pid=80540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:17:45.838660 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-316814f1-7d35-4ae4-9298-0b73d9f66a69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:17:45.844287 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562aee3b-9558-40ef-b14f-1fb64490afbe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a00b3fa7497456895e5e0bf105c4fc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd70d25bf69314820b8aaecb65576978a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:17:45.845135 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562aee3b-9558-40ef-b14f-1fb64490afbe service neutron] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:17:45.845513 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562aee3b-9558-40ef-b14f-1fb64490afbe service neutron] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:17:45.846051 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562aee3b-9558-40ef-b14f-1fb64490afbe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:17:45.860913 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562aee3b-9558-40ef-b14f-1fb64490afbe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:17:45.862949 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562aee3b-9558-40ef-b14f-1fb64490afbe service neutron] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:17:45.874123 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:17:45.875863 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 779/1577] 10.222.0.23 () {60 vars in 1365 bytes} [Thu May 19 12:17:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:17:45.876761 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562aee3b-9558-40ef-b14f-1fb64490afbe service neutron] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:17:45.887380 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562aee3b-9558-40ef-b14f-1fb64490afbe service neutron] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:17:45.893742 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1ad472-ce8f-4727-8e5c-396499b02410 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:17:45.894586 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1ad472-ce8f-4727-8e5c-396499b02410 admin admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:17:45.895581 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1ad472-ce8f-4727-8e5c-396499b02410 admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:17:45.896011 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1ad472-ce8f-4727-8e5c-396499b02410 admin admin] PATH_INFO: `/v3/roles` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:17:45.896890 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1ad472-ce8f-4727-8e5c-396499b02410 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:17:45.898990 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1ad472-ce8f-4727-8e5c-396499b02410 admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:17:45.910654 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 780/1578] 10.222.0.23 () {58 vars in 1121 bytes} [Thu May 19 12:17:45 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 12:17:45.911591 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1ad472-ce8f-4727-8e5c-396499b02410 admin admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:17:45.923379 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1ad472-ce8f-4727-8e5c-396499b02410 admin admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:17:45.928209 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cadcd0ff-fd37-4b3e-947d-34cf9d5ad49d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:17:45.929663 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadcd0ff-fd37-4b3e-947d-34cf9d5ad49d admin admin] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:17:45.929868 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadcd0ff-fd37-4b3e-947d-34cf9d5ad49d admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:17:45.930191 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadcd0ff-fd37-4b3e-947d-34cf9d5ad49d admin admin] PATH_INFO: `/v3/projects` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:17:45.930978 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cadcd0ff-fd37-4b3e-947d-34cf9d5ad49d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:17:45.932573 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 799/1579] 10.222.0.23 () {58 vars in 957 bytes} [Thu May 19 12:17:45 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:17:45.933742 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316814f1-7d35-4ae4-9298-0b73d9f66a69 None None] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:17:45.936496 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cadcd0ff-fd37-4b3e-947d-34cf9d5ad49d admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:17:45.945149 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316814f1-7d35-4ae4-9298-0b73d9f66a69 None None] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:17:45.950847 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f94c61-012b-4450-8cb1-730914039789 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:17:45.951569 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f94c61-012b-4450-8cb1-730914039789 admin admin] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:17:45.951894 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f94c61-012b-4450-8cb1-730914039789 admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:17:45.952176 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f94c61-012b-4450-8cb1-730914039789 admin admin] PATH_INFO: `/v3/users` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:17:45.952877 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f94c61-012b-4450-8cb1-730914039789 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:17:45.954834 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f94c61-012b-4450-8cb1-730914039789 admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:17:46.021998 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 781/1580] 10.222.0.23 () {60 vars in 1152 bytes} [Thu May 19 12:17:45 2022] POST /identity/v3/projects => generated 377 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 12:17:46.024968 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cadcd0ff-fd37-4b3e-947d-34cf9d5ad49d admin admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:17:46.034225 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:17:46.052761 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 800/1581] 10.222.0.23 () {60 vars in 1143 bytes} [Thu May 19 12:17:45 2022] POST /identity/v3/users => generated 288 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 12:17:46.059747 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f94c61-012b-4450-8cb1-730914039789 admin admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:17:46.069096 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f94c61-012b-4450-8cb1-730914039789 admin admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:17:46.095179 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cadcd0ff-fd37-4b3e-947d-34cf9d5ad49d admin admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:17:46.100225 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173d1843-afe8-41d7-a581-2c6b9c340b91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:17:46.100941 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173d1843-afe8-41d7-a581-2c6b9c340b91 admin admin] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:17:46.101231 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173d1843-afe8-41d7-a581-2c6b9c340b91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:17:46.101516 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173d1843-afe8-41d7-a581-2c6b9c340b91 admin admin] PATH_INFO: `/v3/users` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:17:46.102220 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173d1843-afe8-41d7-a581-2c6b9c340b91 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:17:46.104215 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173d1843-afe8-41d7-a581-2c6b9c340b91 admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:17:46.133857 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac8e78f-77d6-4937-9a33-b6c991be9e6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:17:46.134540 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac8e78f-77d6-4937-9a33-b6c991be9e6a admin admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:17:46.134832 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac8e78f-77d6-4937-9a33-b6c991be9e6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:17:46.135128 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac8e78f-77d6-4937-9a33-b6c991be9e6a admin admin] PATH_INFO: `/v3/domains` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:17:46.135889 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac8e78f-77d6-4937-9a33-b6c991be9e6a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:17:46.136223 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac8e78f-77d6-4937-9a33-b6c991be9e6a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:17:46.137871 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac8e78f-77d6-4937-9a33-b6c991be9e6a admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:17:46.150764 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 801/1582] 10.222.0.23 () {58 vars in 1152 bytes} [Thu May 19 12:17:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:17:46.151647 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac8e78f-77d6-4937-9a33-b6c991be9e6a admin admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:17:46.161824 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac8e78f-77d6-4937-9a33-b6c991be9e6a admin admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:17:46.170741 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5d8ffa-f188-49f4-aa68-875a8a00e65b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:17:46.171508 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5d8ffa-f188-49f4-aa68-875a8a00e65b admin admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:17:46.171913 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5d8ffa-f188-49f4-aa68-875a8a00e65b admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:17:46.172268 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5d8ffa-f188-49f4-aa68-875a8a00e65b admin admin] PATH_INFO: `/v3/domains` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:17:46.173542 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5d8ffa-f188-49f4-aa68-875a8a00e65b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:17:46.173542 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5d8ffa-f188-49f4-aa68-875a8a00e65b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:17:46.175359 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5d8ffa-f188-49f4-aa68-875a8a00e65b admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:17:46.190520 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 782/1583] 10.222.0.23 () {60 vars in 1143 bytes} [Thu May 19 12:17:46 2022] POST /identity/v3/users => generated 291 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 12:17:46.191288 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173d1843-afe8-41d7-a581-2c6b9c340b91 admin admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:17:46.203052 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 802/1584] 10.222.0.23 () {58 vars in 1152 bytes} [Thu May 19 12:17:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:17:46.203052 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5d8ffa-f188-49f4-aa68-875a8a00e65b admin admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:17:46.206823 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173d1843-afe8-41d7-a581-2c6b9c340b91 admin admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:17:46.209919 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5d8ffa-f188-49f4-aa68-875a8a00e65b admin admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:17:46.216848 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d96324-1d97-4b73-9d29-593293cc60e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:17:46.218740 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f4e8c4-9aec-4b73-8b21-58685fdbcb81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:17:46.218931 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f4e8c4-9aec-4b73-8b21-58685fdbcb81 admin admin] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:17:46.219345 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f4e8c4-9aec-4b73-8b21-58685fdbcb81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:17:46.219827 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f4e8c4-9aec-4b73-8b21-58685fdbcb81 admin admin] PATH_INFO: `/v3/projects` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:17:46.220715 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f4e8c4-9aec-4b73-8b21-58685fdbcb81 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:17:46.223443 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d96324-1d97-4b73-9d29-593293cc60e6 admin admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:17:46.223916 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d96324-1d97-4b73-9d29-593293cc60e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:17:46.224293 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d96324-1d97-4b73-9d29-593293cc60e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:17:46.225264 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f4e8c4-9aec-4b73-8b21-58685fdbcb81 admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:17:46.225842 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d96324-1d97-4b73-9d29-593293cc60e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:17:46.227494 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d96324-1d97-4b73-9d29-593293cc60e6 admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:17:46.250057 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 783/1585] 10.222.0.23 () {58 vars in 1121 bytes} [Thu May 19 12:17:46 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 12:17:46.251453 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d96324-1d97-4b73-9d29-593293cc60e6 admin admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:17:46.274745 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d96324-1d97-4b73-9d29-593293cc60e6 admin admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:17:46.284314 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c435c4-41f5-4e51-ab40-056a9b7371fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:17:46.285537 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c435c4-41f5-4e51-ab40-056a9b7371fc admin admin] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:17:46.286121 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c435c4-41f5-4e51-ab40-056a9b7371fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:17:46.286635 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c435c4-41f5-4e51-ab40-056a9b7371fc admin admin] PATH_INFO: `/v3/users` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:17:46.289106 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c435c4-41f5-4e51-ab40-056a9b7371fc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:17:46.293280 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c435c4-41f5-4e51-ab40-056a9b7371fc admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:17:46.333184 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 803/1586] 10.222.0.23 () {60 vars in 1152 bytes} [Thu May 19 12:17:46 2022] POST /identity/v3/projects => generated 387 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 12:17:46.333802 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f4e8c4-9aec-4b73-8b21-58685fdbcb81 admin admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:17:46.404525 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 784/1587] 10.222.0.23 () {60 vars in 1143 bytes} [Thu May 19 12:17:46 2022] POST /identity/v3/users => generated 289 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 12:17:46.404525 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c435c4-41f5-4e51-ab40-056a9b7371fc admin admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:17:46.404825 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f4e8c4-9aec-4b73-8b21-58685fdbcb81 admin admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:17:46.407735 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7055242-08fd-4e8a-876d-6a4d4e3a265a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:17:46.408450 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7055242-08fd-4e8a-876d-6a4d4e3a265a admin admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:17:46.408738 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7055242-08fd-4e8a-876d-6a4d4e3a265a admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:17:46.409481 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7055242-08fd-4e8a-876d-6a4d4e3a265a admin admin] PATH_INFO: `/v3/roles` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:17:46.409776 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7055242-08fd-4e8a-876d-6a4d4e3a265a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:17:46.411418 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7055242-08fd-4e8a-876d-6a4d4e3a265a admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:17:46.412951 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c435c4-41f5-4e51-ab40-056a9b7371fc admin admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:17:46.418768 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c0590d-9659-4ea5-b2c2-49eb61f80f0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:17:46.419448 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c0590d-9659-4ea5-b2c2-49eb61f80f0f admin admin] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:17:46.419738 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c0590d-9659-4ea5-b2c2-49eb61f80f0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:17:46.420078 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c0590d-9659-4ea5-b2c2-49eb61f80f0f admin admin] PATH_INFO: `/v3/projects` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:17:46.420781 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c0590d-9659-4ea5-b2c2-49eb61f80f0f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:17:46.422931 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c0590d-9659-4ea5-b2c2-49eb61f80f0f admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:17:46.423064 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 804/1588] 10.222.0.23 () {58 vars in 1121 bytes} [Thu May 19 12:17:46 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 12:17:46.425763 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7055242-08fd-4e8a-876d-6a4d4e3a265a admin admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:17:46.434864 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:17:46.440715 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7055242-08fd-4e8a-876d-6a4d4e3a265a admin admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:17:46.445586 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ed4e74-d505-4309-a284-7a138f932851 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:17:46.446359 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ed4e74-d505-4309-a284-7a138f932851 admin admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:17:46.447771 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ed4e74-d505-4309-a284-7a138f932851 admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:17:46.451004 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ed4e74-d505-4309-a284-7a138f932851 admin admin] PATH_INFO: `/v3/roles` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:17:46.451004 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ed4e74-d505-4309-a284-7a138f932851 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:17:46.451004 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ed4e74-d505-4309-a284-7a138f932851 admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:17:46.462437 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 805/1589] 10.222.0.23 () {58 vars in 1121 bytes} [Thu May 19 12:17:46 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 12:17:46.463312 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ed4e74-d505-4309-a284-7a138f932851 admin admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:17:46.473989 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ed4e74-d505-4309-a284-7a138f932851 admin admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:17:46.479739 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2218b14-ffa6-46c5-9d48-21365097b210 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:17:46.483942 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2218b14-ffa6-46c5-9d48-21365097b210 admin admin] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:17:46.483942 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2218b14-ffa6-46c5-9d48-21365097b210 admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:17:46.483942 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2218b14-ffa6-46c5-9d48-21365097b210 admin admin] PATH_INFO: `/v3/projects` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:17:46.483942 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2218b14-ffa6-46c5-9d48-21365097b210 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:17:46.485175 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2218b14-ffa6-46c5-9d48-21365097b210 admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:17:46.541961 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 785/1590] 10.222.0.23 () {60 vars in 1152 bytes} [Thu May 19 12:17:46 2022] POST /identity/v3/projects => generated 365 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 12:17:46.542961 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c0590d-9659-4ea5-b2c2-49eb61f80f0f admin admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:17:46.606245 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 806/1591] 10.222.0.23 () {60 vars in 1152 bytes} [Thu May 19 12:17:46 2022] POST /identity/v3/projects => generated 371 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 12:17:46.606995 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c0590d-9659-4ea5-b2c2-49eb61f80f0f admin admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:17:46.607242 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2218b14-ffa6-46c5-9d48-21365097b210 admin admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:17:46.673471 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:17:46.682291 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2218b14-ffa6-46c5-9d48-21365097b210 admin admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:17:46.683608 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da6b5ff-7153-4054-8535-c1dccb33aab1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:17:46.685929 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da6b5ff-7153-4054-8535-c1dccb33aab1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:17:46.685929 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da6b5ff-7153-4054-8535-c1dccb33aab1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:17:46.685929 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da6b5ff-7153-4054-8535-c1dccb33aab1 admin admin] PATH_INFO: `/v3/projects/de8f3490bba84ff38b5940fe768ede25/users/9e66a142d53340bb8bc4a0a9a9f48d59/roles/259eb400cdb4439db72d14456ae9cd08` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:17:46.688308 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f529655f-ce61-4043-ba4b-7c20a16a8778 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:17:46.689000 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f529655f-ce61-4043-ba4b-7c20a16a8778 admin admin] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:17:46.689282 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f529655f-ce61-4043-ba4b-7c20a16a8778 admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:17:46.689568 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f529655f-ce61-4043-ba4b-7c20a16a8778 admin admin] PATH_INFO: `/v3/users` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:17:46.690268 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f529655f-ce61-4043-ba4b-7c20a16a8778 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:17:46.692390 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f529655f-ce61-4043-ba4b-7c20a16a8778 admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:17:46.703499 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da6b5ff-7153-4054-8535-c1dccb33aab1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=de8f3490bba84ff38b5940fe768ede25, user_id=9e66a142d53340bb8bc4a0a9a9f48d59, role_id=259eb400cdb4439db72d14456ae9cd08)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:17:46.706279 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da6b5ff-7153-4054-8535-c1dccb33aab1 admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:17:46.771605 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 807/1592] 10.222.0.23 () {60 vars in 1143 bytes} [Thu May 19 12:17:46 2022] POST /identity/v3/users => generated 300 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 12:17:46.771605 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f529655f-ce61-4043-ba4b-7c20a16a8778 admin admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:17:46.780440 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f529655f-ce61-4043-ba4b-7c20a16a8778 admin admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:17:46.785181 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13975c45-e872-44cb-b680-e88eaa253f9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:17:46.785865 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13975c45-e872-44cb-b680-e88eaa253f9e admin admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:17:46.786151 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13975c45-e872-44cb-b680-e88eaa253f9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:17:46.786419 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13975c45-e872-44cb-b680-e88eaa253f9e admin admin] PATH_INFO: `/v3/roles` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:17:46.787159 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13975c45-e872-44cb-b680-e88eaa253f9e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:17:46.789466 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13975c45-e872-44cb-b680-e88eaa253f9e admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:17:46.799171 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 786/1593] 10.222.0.23 () {60 vars in 1482 bytes} [Thu May 19 12:17:46 2022] PUT /identity/v3/projects/de8f3490bba84ff38b5940fe768ede25/users/9e66a142d53340bb8bc4a0a9a9f48d59/roles/259eb400cdb4439db72d14456ae9cd08 => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:17:46.801988 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da6b5ff-7153-4054-8535-c1dccb33aab1 admin admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:17:46.802107 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 808/1594] 10.222.0.23 () {58 vars in 1121 bytes} [Thu May 19 12:17:46 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 12:17:46.802107 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13975c45-e872-44cb-b680-e88eaa253f9e admin admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:17:46.812154 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13975c45-e872-44cb-b680-e88eaa253f9e admin admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:17:46.876542 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da6b5ff-7153-4054-8535-c1dccb33aab1 admin admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:17:46.881371 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825c2b1f-96bf-49f9-baa1-9deebd14cfb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:17:46.882127 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825c2b1f-96bf-49f9-baa1-9deebd14cfb7 admin admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:17:46.882406 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825c2b1f-96bf-49f9-baa1-9deebd14cfb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:17:46.882678 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825c2b1f-96bf-49f9-baa1-9deebd14cfb7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:17:46.883602 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825c2b1f-96bf-49f9-baa1-9deebd14cfb7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:17:46.884494 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99600c3a-e32e-48a4-80ce-c1e53064492d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:17:46.885250 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825c2b1f-96bf-49f9-baa1-9deebd14cfb7 admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:17:46.885375 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99600c3a-e32e-48a4-80ce-c1e53064492d admin admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:17:46.885541 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99600c3a-e32e-48a4-80ce-c1e53064492d admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:17:46.885827 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99600c3a-e32e-48a4-80ce-c1e53064492d admin admin] PATH_INFO: `/v3/roles` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:17:46.886551 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99600c3a-e32e-48a4-80ce-c1e53064492d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:17:46.888373 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99600c3a-e32e-48a4-80ce-c1e53064492d admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:17:46.900665 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 787/1595] 10.222.0.23 () {58 vars in 1121 bytes} [Thu May 19 12:17:46 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 12:17:46.904607 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 809/1596] 10.222.0.23 () {58 vars in 1121 bytes} [Thu May 19 12:17:46 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 12:17:46.904760 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825c2b1f-96bf-49f9-baa1-9deebd14cfb7 admin admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:17:46.904845 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99600c3a-e32e-48a4-80ce-c1e53064492d admin admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:17:46.915809 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99600c3a-e32e-48a4-80ce-c1e53064492d admin admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:17:46.919766 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825c2b1f-96bf-49f9-baa1-9deebd14cfb7 admin admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:17:46.921116 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe739a3-156a-4bf6-9b7b-ea3d8b1f4e56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:17:46.922024 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe739a3-156a-4bf6-9b7b-ea3d8b1f4e56 admin admin] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:17:46.922325 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe739a3-156a-4bf6-9b7b-ea3d8b1f4e56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:17:46.922756 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe739a3-156a-4bf6-9b7b-ea3d8b1f4e56 admin admin] PATH_INFO: `/v3/users` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:17:46.923519 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe739a3-156a-4bf6-9b7b-ea3d8b1f4e56 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:17:46.925127 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc74aa4-8041-49d7-b6c9-e571bb453013 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:17:46.926076 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc74aa4-8041-49d7-b6c9-e571bb453013 admin admin] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:17:46.926373 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc74aa4-8041-49d7-b6c9-e571bb453013 admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:17:46.926657 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc74aa4-8041-49d7-b6c9-e571bb453013 admin admin] PATH_INFO: `/v3/users` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:17:46.928861 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc74aa4-8041-49d7-b6c9-e571bb453013 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:17:46.931585 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe739a3-156a-4bf6-9b7b-ea3d8b1f4e56 admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:17:46.933134 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc74aa4-8041-49d7-b6c9-e571bb453013 admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:17:47.016406 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 788/1597] 10.222.0.23 () {60 vars in 1143 bytes} [Thu May 19 12:17:46 2022] POST /identity/v3/users => generated 283 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 12:17:47.017409 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc74aa4-8041-49d7-b6c9-e571bb453013 admin admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:17:47.030167 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc74aa4-8041-49d7-b6c9-e571bb453013 admin admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:17:47.035321 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2662680e-304d-4973-999d-0212cdb1a747 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:17:47.036092 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2662680e-304d-4973-999d-0212cdb1a747 admin admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:17:47.036440 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2662680e-304d-4973-999d-0212cdb1a747 admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:17:47.036723 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2662680e-304d-4973-999d-0212cdb1a747 admin admin] PATH_INFO: `/v3/roles` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:17:47.037493 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2662680e-304d-4973-999d-0212cdb1a747 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:17:47.039994 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2662680e-304d-4973-999d-0212cdb1a747 admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:17:47.056373 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 810/1598] 10.222.0.23 () {60 vars in 1143 bytes} [Thu May 19 12:17:46 2022] POST /identity/v3/users => generated 286 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 12:17:47.057548 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 789/1599] 10.222.0.23 () {58 vars in 1121 bytes} [Thu May 19 12:17:47 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 12:17:47.058760 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2662680e-304d-4973-999d-0212cdb1a747 admin admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:17:47.060719 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d0e791-f862-40ce-b688-ca4915e6914d None None] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:17:47.061123 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d0e791-f862-40ce-b688-ca4915e6914d None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:17:47.061614 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d0e791-f862-40ce-b688-ca4915e6914d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:17:47.076599 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2662680e-304d-4973-999d-0212cdb1a747 admin admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:17:47.081834 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4786199d-e8d3-4026-bcd3-971c9bcbc0d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:17:47.087260 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4786199d-e8d3-4026-bcd3-971c9bcbc0d0 admin admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:17:47.087260 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4786199d-e8d3-4026-bcd3-971c9bcbc0d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:17:47.087260 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4786199d-e8d3-4026-bcd3-971c9bcbc0d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:17:47.087260 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4786199d-e8d3-4026-bcd3-971c9bcbc0d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:17:47.087260 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4786199d-e8d3-4026-bcd3-971c9bcbc0d0 admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:17:47.092021 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.auth.core [None req-90d0e791-f862-40ce-b688-ca4915e6914d None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:17:47.094157 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-90d0e791-f862-40ce-b688-ca4915e6914d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:17:47.097785 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 790/1600] 10.222.0.23 () {58 vars in 1121 bytes} [Thu May 19 12:17:47 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 12:17:47.098622 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4786199d-e8d3-4026-bcd3-971c9bcbc0d0 admin admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:17:47.116038 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 811/1601] 10.222.0.23 () {58 vars in 957 bytes} [Thu May 19 12:17:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:17:47.116038 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d0e791-f862-40ce-b688-ca4915e6914d None None] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:17:47.117343 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4786199d-e8d3-4026-bcd3-971c9bcbc0d0 admin admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:17:47.117343 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b42472c-e0fe-449f-98d2-0a78d3787007 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:17:47.117343 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b42472c-e0fe-449f-98d2-0a78d3787007 admin admin] REQUEST_METHOD: `PUT` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:17:47.117985 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b42472c-e0fe-449f-98d2-0a78d3787007 admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:17:47.118289 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b42472c-e0fe-449f-98d2-0a78d3787007 admin admin] PATH_INFO: `/v3/projects/5a530cf15e11460b898957828f7d858e/users/ec3d521695a64c62a6a2844594ac420e/roles/259eb400cdb4439db72d14456ae9cd08` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:17:47.126239 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d0e791-f862-40ce-b688-ca4915e6914d None None] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:17:47.131609 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec537ae9-f111-471f-ae19-93bb7ee7d0c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:17:47.133525 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec537ae9-f111-471f-ae19-93bb7ee7d0c7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:17:47.133781 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec537ae9-f111-471f-ae19-93bb7ee7d0c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:17:47.134104 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec537ae9-f111-471f-ae19-93bb7ee7d0c7 admin admin] PATH_INFO: `/v3/projects/1d2de86d602945238f5df0e2c388004f/users/dd39ddcf4ab045b98a737ee529a2dcce/roles/259eb400cdb4439db72d14456ae9cd08` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:17:47.144516 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b42472c-e0fe-449f-98d2-0a78d3787007 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5a530cf15e11460b898957828f7d858e, user_id=ec3d521695a64c62a6a2844594ac420e, role_id=259eb400cdb4439db72d14456ae9cd08)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:17:47.147209 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b42472c-e0fe-449f-98d2-0a78d3787007 admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:17:47.152096 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec537ae9-f111-471f-ae19-93bb7ee7d0c7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1d2de86d602945238f5df0e2c388004f, user_id=dd39ddcf4ab045b98a737ee529a2dcce, role_id=259eb400cdb4439db72d14456ae9cd08)` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:17:47.153216 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: WARNING py.warnings [None req-ec537ae9-f111-471f-ae19-93bb7ee7d0c7 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 19 12:17:47.153216 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: warnings.warn(msg) May 19 12:17:47.153216 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: May 19 12:17:47.155211 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec537ae9-f111-471f-ae19-93bb7ee7d0c7 admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:17:47.158675 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:17:47.262328 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 791/1602] 10.222.0.23 () {60 vars in 1482 bytes} [Thu May 19 12:17:47 2022] PUT /identity/v3/projects/5a530cf15e11460b898957828f7d858e/users/ec3d521695a64c62a6a2844594ac420e/roles/259eb400cdb4439db72d14456ae9cd08 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:17:47.277619 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b42472c-e0fe-449f-98d2-0a78d3787007 admin admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:17:47.306726 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 812/1603] 10.222.0.23 () {60 vars in 1482 bytes} [Thu May 19 12:17:47 2022] PUT /identity/v3/projects/1d2de86d602945238f5df0e2c388004f/users/dd39ddcf4ab045b98a737ee529a2dcce/roles/259eb400cdb4439db72d14456ae9cd08 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:17:47.308533 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec537ae9-f111-471f-ae19-93bb7ee7d0c7 admin admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:17:47.390172 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b42472c-e0fe-449f-98d2-0a78d3787007 admin admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:17:47.396490 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebd217b-459a-41d1-8e31-874e103190b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:17:47.397191 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebd217b-459a-41d1-8e31-874e103190b3 admin admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:17:47.397549 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebd217b-459a-41d1-8e31-874e103190b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:17:47.397753 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec537ae9-f111-471f-ae19-93bb7ee7d0c7 admin admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:17:47.397976 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebd217b-459a-41d1-8e31-874e103190b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:17:47.398896 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ebd217b-459a-41d1-8e31-874e103190b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:17:47.400800 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ebd217b-459a-41d1-8e31-874e103190b3 admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:17:47.403378 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b9960b-f3cd-46d5-90ff-b0563ebc029a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:17:47.404176 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b9960b-f3cd-46d5-90ff-b0563ebc029a admin admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:17:47.410334 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b9960b-f3cd-46d5-90ff-b0563ebc029a admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:17:47.410334 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b9960b-f3cd-46d5-90ff-b0563ebc029a admin admin] PATH_INFO: `/v3/roles` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:17:47.410334 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b9960b-f3cd-46d5-90ff-b0563ebc029a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:17:47.410334 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b9960b-f3cd-46d5-90ff-b0563ebc029a admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:17:47.416189 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 792/1604] 10.222.0.23 () {58 vars in 1121 bytes} [Thu May 19 12:17:47 2022] GET /identity/v3/roles => generated 1420 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 12:17:47.417396 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebd217b-459a-41d1-8e31-874e103190b3 admin admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:17:47.423678 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 813/1605] 10.222.0.23 () {58 vars in 1121 bytes} [Thu May 19 12:17:47 2022] GET /identity/v3/roles => generated 1420 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 12:17:47.423678 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b9960b-f3cd-46d5-90ff-b0563ebc029a admin admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:17:47.436663 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebd217b-459a-41d1-8e31-874e103190b3 admin admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:17:47.446860 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b9960b-f3cd-46d5-90ff-b0563ebc029a admin admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:17:47.447015 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d03485d-7526-4872-bb88-be69c4d5579e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:17:47.447015 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d03485d-7526-4872-bb88-be69c4d5579e admin admin] REQUEST_METHOD: `PUT` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:17:47.447015 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d03485d-7526-4872-bb88-be69c4d5579e admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:17:47.447015 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d03485d-7526-4872-bb88-be69c4d5579e admin admin] PATH_INFO: `/v3/projects/5468e69ac0c843f4b31fda5934c88728/users/01aa7be0f56246be8ce9c1aa72567851/roles/9db5671c65d3414ea2682afa2ccf1403` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:17:47.449845 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:17:47.452121 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c150a4c-b5f3-4bca-834a-d4e8a91041b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:17:47.455343 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c150a4c-b5f3-4bca-834a-d4e8a91041b2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:17:47.456426 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c150a4c-b5f3-4bca-834a-d4e8a91041b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:17:47.456968 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c150a4c-b5f3-4bca-834a-d4e8a91041b2 admin admin] PATH_INFO: `/v3/projects/9eafee25cfb14ca2bd5c10fe41a01644/users/9d762cce00ae4e468c358b0437f35deb/roles/259eb400cdb4439db72d14456ae9cd08` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:17:47.479649 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c150a4c-b5f3-4bca-834a-d4e8a91041b2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9eafee25cfb14ca2bd5c10fe41a01644, user_id=9d762cce00ae4e468c358b0437f35deb, role_id=259eb400cdb4439db72d14456ae9cd08)` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:17:47.482753 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d03485d-7526-4872-bb88-be69c4d5579e admin admin] RBAC: Authorizing `identity:create_grant(project_id=5468e69ac0c843f4b31fda5934c88728, user_id=01aa7be0f56246be8ce9c1aa72567851, role_id=9db5671c65d3414ea2682afa2ccf1403)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:17:47.484664 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c150a4c-b5f3-4bca-834a-d4e8a91041b2 admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:17:47.489237 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d03485d-7526-4872-bb88-be69c4d5579e admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:17:47.566626 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 814/1606] 10.222.0.23 () {60 vars in 1482 bytes} [Thu May 19 12:17:47 2022] PUT /identity/v3/projects/9eafee25cfb14ca2bd5c10fe41a01644/users/9d762cce00ae4e468c358b0437f35deb/roles/259eb400cdb4439db72d14456ae9cd08 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:17:47.568015 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c150a4c-b5f3-4bca-834a-d4e8a91041b2 admin admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:17:47.603392 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 793/1607] 10.222.0.23 () {60 vars in 1482 bytes} [Thu May 19 12:17:47 2022] PUT /identity/v3/projects/5468e69ac0c843f4b31fda5934c88728/users/01aa7be0f56246be8ce9c1aa72567851/roles/9db5671c65d3414ea2682afa2ccf1403 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:17:47.604084 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8e0d2a-7d5f-4cef-9cfa-404f8070910d None None] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:17:47.604273 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8e0d2a-7d5f-4cef-9cfa-404f8070910d None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:17:47.604558 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8e0d2a-7d5f-4cef-9cfa-404f8070910d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:17:47.648577 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c150a4c-b5f3-4bca-834a-d4e8a91041b2 admin admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:17:47.650090 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.auth.core [None req-ad8e0d2a-7d5f-4cef-9cfa-404f8070910d None None] MFA Rules not processed for user `ec3d521695a64c62a6a2844594ac420e`. Rule list: `[]` (Enabled: `True`). {{(pid=80539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:17:47.651667 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-ad8e0d2a-7d5f-4cef-9cfa-404f8070910d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:17:47.717069 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d29db8e-3c1d-4e9b-b751-e79fcda76bf0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a00b3fa7497456895e5e0bf105c4fc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd70d25bf69314820b8aaecb65576978a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:17:47.718371 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d29db8e-3c1d-4e9b-b751-e79fcda76bf0 service neutron] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:17:47.719377 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d29db8e-3c1d-4e9b-b751-e79fcda76bf0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:17:47.719377 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d29db8e-3c1d-4e9b-b751-e79fcda76bf0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:17:47.739062 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 794/1608] 10.222.0.23 () {58 vars in 957 bytes} [Thu May 19 12:17:47 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:17:47.740543 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94965673-acb7-486d-b8ee-9c43f1776189 None None] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:17:47.740831 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94965673-acb7-486d-b8ee-9c43f1776189 None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:17:47.741107 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94965673-acb7-486d-b8ee-9c43f1776189 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:17:47.781321 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.auth.core [None req-94965673-acb7-486d-b8ee-9c43f1776189 None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:17:47.782842 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-94965673-acb7-486d-b8ee-9c43f1776189 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:17:47.788460 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:17:47.825585 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d29db8e-3c1d-4e9b-b751-e79fcda76bf0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:17:47.827856 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d29db8e-3c1d-4e9b-b751-e79fcda76bf0 service neutron] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:17:47.840610 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 815/1609] 10.222.0.23 () {60 vars in 1365 bytes} [Thu May 19 12:17:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 273 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:17:47.841520 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d29db8e-3c1d-4e9b-b751-e79fcda76bf0 service neutron] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:17:47.855104 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d29db8e-3c1d-4e9b-b751-e79fcda76bf0 service neutron] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:17:47.868593 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ff03d9-7c70-4cd2-b143-38a712dc3d13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:17:47.869527 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ff03d9-7c70-4cd2-b143-38a712dc3d13 admin admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:17:47.870698 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ff03d9-7c70-4cd2-b143-38a712dc3d13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:17:47.870976 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ff03d9-7c70-4cd2-b143-38a712dc3d13 admin admin] PATH_INFO: `/v3/roles` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:17:47.871912 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ff03d9-7c70-4cd2-b143-38a712dc3d13 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:17:47.874007 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ff03d9-7c70-4cd2-b143-38a712dc3d13 admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:17:47.891857 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 816/1610] 10.222.0.23 () {58 vars in 1121 bytes} [Thu May 19 12:17:47 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 12:17:47.893706 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ff03d9-7c70-4cd2-b143-38a712dc3d13 admin admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:17:47.894871 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 795/1611] 10.222.0.23 () {58 vars in 957 bytes} [Thu May 19 12:17:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:17:47.895887 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0d429f-7041-464d-94d3-389ed52b0295 None None] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:17:47.896284 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0d429f-7041-464d-94d3-389ed52b0295 None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:17:47.896878 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0d429f-7041-464d-94d3-389ed52b0295 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:17:47.907960 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ff03d9-7c70-4cd2-b143-38a712dc3d13 admin admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:17:47.913893 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798b09f7-5a3c-41e9-a641-309d835f1866 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:17:47.914932 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798b09f7-5a3c-41e9-a641-309d835f1866 admin admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:17:47.915159 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798b09f7-5a3c-41e9-a641-309d835f1866 admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:17:47.915986 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798b09f7-5a3c-41e9-a641-309d835f1866 admin admin] PATH_INFO: `/v3/roles` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:17:47.916456 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798b09f7-5a3c-41e9-a641-309d835f1866 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:17:47.918593 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798b09f7-5a3c-41e9-a641-309d835f1866 admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:17:47.928044 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.auth.core [None req-ff0d429f-7041-464d-94d3-389ed52b0295 None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:17:47.931183 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-ff0d429f-7041-464d-94d3-389ed52b0295 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:17:47.937891 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 817/1612] 10.222.0.23 () {58 vars in 1121 bytes} [Thu May 19 12:17:47 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 12:17:47.938621 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798b09f7-5a3c-41e9-a641-309d835f1866 admin admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:17:47.948052 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 796/1613] 10.222.0.23 () {58 vars in 957 bytes} [Thu May 19 12:17:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:17:47.951738 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f26e1a1-6cc0-42fd-b49d-7384e783a46d None None] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:17:47.961082 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f26e1a1-6cc0-42fd-b49d-7384e783a46d None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:17:47.961082 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f26e1a1-6cc0-42fd-b49d-7384e783a46d None None] PATH_INFO: `/` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:17:47.961280 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798b09f7-5a3c-41e9-a641-309d835f1866 admin admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:17:47.965563 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 797/1614] 10.222.0.23 () {54 vars in 961 bytes} [Thu May 19 12:17:47 2022] GET /identity => generated 270 bytes in 17 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 12:17:47.966644 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f26e1a1-6cc0-42fd-b49d-7384e783a46d None None] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:17:47.971838 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de14a2f-ba11-4bfa-bca5-a89337e63dee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:17:47.972743 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de14a2f-ba11-4bfa-bca5-a89337e63dee admin admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:17:47.973030 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de14a2f-ba11-4bfa-bca5-a89337e63dee admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:17:47.973312 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de14a2f-ba11-4bfa-bca5-a89337e63dee admin admin] PATH_INFO: `/v3/roles` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:17:47.974060 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de14a2f-ba11-4bfa-bca5-a89337e63dee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:17:47.977366 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de14a2f-ba11-4bfa-bca5-a89337e63dee admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:17:47.987343 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f26e1a1-6cc0-42fd-b49d-7384e783a46d None None] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:17:47.997519 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 818/1615] 10.222.0.23 () {58 vars in 1121 bytes} [Thu May 19 12:17:47 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 12:17:47.998644 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de14a2f-ba11-4bfa-bca5-a89337e63dee admin admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:17:48.000824 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195ebf61-44c1-4b31-99d2-f636cebd48bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:17:48.001968 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195ebf61-44c1-4b31-99d2-f636cebd48bb admin admin] REQUEST_METHOD: `PUT` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:17:48.002350 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195ebf61-44c1-4b31-99d2-f636cebd48bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:17:48.005541 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195ebf61-44c1-4b31-99d2-f636cebd48bb admin admin] PATH_INFO: `/v3/projects/4c4d3055c4f544ada2d283b6f8e95405/users/15a744f7a0f042258579fb0a3600f612/roles/259eb400cdb4439db72d14456ae9cd08` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:17:48.015808 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de14a2f-ba11-4bfa-bca5-a89337e63dee admin admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:17:48.024395 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7757ed-c1ef-4440-9848-607867dc3433 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a00b3fa7497456895e5e0bf105c4fc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd70d25bf69314820b8aaecb65576978a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:17:48.025137 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7757ed-c1ef-4440-9848-607867dc3433 service neutron] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:17:48.025567 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7757ed-c1ef-4440-9848-607867dc3433 service neutron] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:17:48.026474 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7757ed-c1ef-4440-9848-607867dc3433 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:17:48.032728 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195ebf61-44c1-4b31-99d2-f636cebd48bb admin admin] RBAC: Authorizing `identity:create_grant(project_id=4c4d3055c4f544ada2d283b6f8e95405, user_id=15a744f7a0f042258579fb0a3600f612, role_id=259eb400cdb4439db72d14456ae9cd08)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:17:48.036065 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195ebf61-44c1-4b31-99d2-f636cebd48bb admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:17:48.049112 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7757ed-c1ef-4440-9848-607867dc3433 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:17:48.051853 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7757ed-c1ef-4440-9848-607867dc3433 service neutron] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:17:48.066643 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 819/1616] 10.222.0.23 () {60 vars in 1365 bytes} [Thu May 19 12:17:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 68 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:17:48.067550 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7757ed-c1ef-4440-9848-607867dc3433 service neutron] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:17:48.079896 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7757ed-c1ef-4440-9848-607867dc3433 service neutron] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:17:48.087448 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304e1a24-f725-453e-8162-9ee872f70eb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:17:48.088746 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304e1a24-f725-453e-8162-9ee872f70eb6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:17:48.089091 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304e1a24-f725-453e-8162-9ee872f70eb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:17:48.095755 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304e1a24-f725-453e-8162-9ee872f70eb6 admin admin] PATH_INFO: `/v3/projects/25ad8063b2a84a529dfa0e5a5f1a8597/users/08994739116f4675b5a9e4e11c42ce5c/roles/259eb400cdb4439db72d14456ae9cd08` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:17:48.113746 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304e1a24-f725-453e-8162-9ee872f70eb6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=25ad8063b2a84a529dfa0e5a5f1a8597, user_id=08994739116f4675b5a9e4e11c42ce5c, role_id=259eb400cdb4439db72d14456ae9cd08)` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:17:48.117565 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304e1a24-f725-453e-8162-9ee872f70eb6 admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:17:48.161388 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 798/1617] 10.222.0.23 () {60 vars in 1482 bytes} [Thu May 19 12:17:47 2022] PUT /identity/v3/projects/4c4d3055c4f544ada2d283b6f8e95405/users/15a744f7a0f042258579fb0a3600f612/roles/259eb400cdb4439db72d14456ae9cd08 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:17:48.161388 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195ebf61-44c1-4b31-99d2-f636cebd48bb admin admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:17:48.256464 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 820/1618] 10.222.0.23 () {60 vars in 1482 bytes} [Thu May 19 12:17:48 2022] PUT /identity/v3/projects/25ad8063b2a84a529dfa0e5a5f1a8597/users/08994739116f4675b5a9e4e11c42ce5c/roles/259eb400cdb4439db72d14456ae9cd08 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:17:48.256464 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78217c5-d3b6-4b58-bb04-9fa055e6bcb1 None None] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:17:48.256464 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78217c5-d3b6-4b58-bb04-9fa055e6bcb1 None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:17:48.256464 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78217c5-d3b6-4b58-bb04-9fa055e6bcb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:17:48.261232 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:17:48.281893 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195ebf61-44c1-4b31-99d2-f636cebd48bb admin admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:17:48.311745 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.auth.core [None req-e78217c5-d3b6-4b58-bb04-9fa055e6bcb1 None None] MFA Rules not processed for user `34a98d75d84b4c67981377e0f26eca62`. Rule list: `[]` (Enabled: `True`). {{(pid=80540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:17:48.313209 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-e78217c5-d3b6-4b58-bb04-9fa055e6bcb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:17:48.365300 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c313b378-81e3-43e9-b411-6d07cee1d030 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a00b3fa7497456895e5e0bf105c4fc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd70d25bf69314820b8aaecb65576978a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:17:48.370159 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c313b378-81e3-43e9-b411-6d07cee1d030 service neutron] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:17:48.370159 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c313b378-81e3-43e9-b411-6d07cee1d030 service neutron] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:17:48.370159 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c313b378-81e3-43e9-b411-6d07cee1d030 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:17:48.456875 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 821/1619] 10.222.0.23 () {58 vars in 1058 bytes} [Thu May 19 12:17:48 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:17:48.458599 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78217c5-d3b6-4b58-bb04-9fa055e6bcb1 None None] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:17:48.475429 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c313b378-81e3-43e9-b411-6d07cee1d030 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:17:48.479644 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c313b378-81e3-43e9-b411-6d07cee1d030 service neutron] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:17:48.497917 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 799/1620] 10.222.0.23 () {60 vars in 1365 bytes} [Thu May 19 12:17:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:17:48.500533 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2083f9-2f24-4bf0-a4fe-698bb465ef00 None None] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:17:48.501239 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2083f9-2f24-4bf0-a4fe-698bb465ef00 None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:17:48.501703 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2083f9-2f24-4bf0-a4fe-698bb465ef00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:17:48.556925 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.auth.core [None req-de2083f9-2f24-4bf0-a4fe-698bb465ef00 None None] MFA Rules not processed for user `15a744f7a0f042258579fb0a3600f612`. Rule list: `[]` (Enabled: `True`). {{(pid=80539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:17:48.586280 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-de2083f9-2f24-4bf0-a4fe-698bb465ef00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:17:48.609052 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:17:48.628876 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78217c5-d3b6-4b58-bb04-9fa055e6bcb1 None None] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:17:48.640424 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfee833-e136-476b-9bf8-056c3e1b27d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:17:48.643986 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfee833-e136-476b-9bf8-056c3e1b27d8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:17:48.644711 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfee833-e136-476b-9bf8-056c3e1b27d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:17:48.645457 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfee833-e136-476b-9bf8-056c3e1b27d8 admin admin] PATH_INFO: `/v3/projects/5468e69ac0c843f4b31fda5934c88728/users/01aa7be0f56246be8ce9c1aa72567851/roles/9db5671c65d3414ea2682afa2ccf1403` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:17:48.652571 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cfee833-e136-476b-9bf8-056c3e1b27d8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5468e69ac0c843f4b31fda5934c88728, user_id=01aa7be0f56246be8ce9c1aa72567851, role_id=9db5671c65d3414ea2682afa2ccf1403)` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:17:48.662167 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cfee833-e136-476b-9bf8-056c3e1b27d8 admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:17:48.698453 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 822/1621] 10.222.0.23 () {60 vars in 1482 bytes} [Thu May 19 12:17:48 2022] PUT /identity/v3/projects/5468e69ac0c843f4b31fda5934c88728/users/01aa7be0f56246be8ce9c1aa72567851/roles/9db5671c65d3414ea2682afa2ccf1403 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:17:48.698453 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec8e773-42b9-4f4b-a059-6d0b879a567f None None] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:17:48.698453 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec8e773-42b9-4f4b-a059-6d0b879a567f None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:17:48.698453 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec8e773-42b9-4f4b-a059-6d0b879a567f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:17:48.748294 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.auth.core [None req-2ec8e773-42b9-4f4b-a059-6d0b879a567f None None] MFA Rules not processed for user `08994739116f4675b5a9e4e11c42ce5c`. Rule list: `[]` (Enabled: `True`). {{(pid=80540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:17:48.753407 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-2ec8e773-42b9-4f4b-a059-6d0b879a567f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:17:48.800266 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 800/1622] 10.222.0.23 () {58 vars in 957 bytes} [Thu May 19 12:17:48 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 295 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:17:48.800266 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2083f9-2f24-4bf0-a4fe-698bb465ef00 None None] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:17:48.836842 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2083f9-2f24-4bf0-a4fe-698bb465ef00 None None] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:17:48.927825 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 823/1623] 10.222.0.23 () {58 vars in 957 bytes} [Thu May 19 12:17:48 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:17:48.927825 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5892b32a-e2d7-44c9-b46f-379ea528ba3f None None] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:17:48.927825 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5892b32a-e2d7-44c9-b46f-379ea528ba3f None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:17:48.927825 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5892b32a-e2d7-44c9-b46f-379ea528ba3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:17:48.974367 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d073ec-55b1-4e08-8a9d-a52be7e57795 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34a98d75d84b4c67981377e0f26eca62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd70d25bf69314820b8aaecb65576978a', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:17:48.975655 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.auth.core [None req-5892b32a-e2d7-44c9-b46f-379ea528ba3f None None] MFA Rules not processed for user `01aa7be0f56246be8ce9c1aa72567851`. Rule list: `[]` (Enabled: `True`). {{(pid=80540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:17:48.976652 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d073ec-55b1-4e08-8a9d-a52be7e57795 service nova] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:17:48.976932 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d073ec-55b1-4e08-8a9d-a52be7e57795 service nova] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:17:48.977424 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d073ec-55b1-4e08-8a9d-a52be7e57795 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:17:48.977799 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-5892b32a-e2d7-44c9-b46f-379ea528ba3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:17:49.113811 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 824/1624] 10.222.0.23 () {58 vars in 957 bytes} [Thu May 19 12:17:48 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:17:49.113811 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5892b32a-e2d7-44c9-b46f-379ea528ba3f None None] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:17:49.114566 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d073ec-55b1-4e08-8a9d-a52be7e57795 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:17:49.115305 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d073ec-55b1-4e08-8a9d-a52be7e57795 service nova] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:17:49.130064 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 801/1625] 10.222.0.23 () {60 vars in 1365 bytes} [Thu May 19 12:17:48 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:17:49.132981 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d073ec-55b1-4e08-8a9d-a52be7e57795 service nova] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:17:49.266017 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d073ec-55b1-4e08-8a9d-a52be7e57795 service nova] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:17:49.279762 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:17:49.282955 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccda7ca8-cf38-4fa7-afb2-87c387f2ad58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:17:49.285732 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccda7ca8-cf38-4fa7-afb2-87c387f2ad58 admin admin] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:17:49.286072 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccda7ca8-cf38-4fa7-afb2-87c387f2ad58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:17:49.286357 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccda7ca8-cf38-4fa7-afb2-87c387f2ad58 admin admin] PATH_INFO: `/v3/projects` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:17:49.287267 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccda7ca8-cf38-4fa7-afb2-87c387f2ad58 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:17:49.292162 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccda7ca8-cf38-4fa7-afb2-87c387f2ad58 admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:17:49.297560 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5892b32a-e2d7-44c9-b46f-379ea528ba3f None None] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:17:49.303747 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab4dcd2-eb1b-495d-b601-b36cdea2514c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8800254c0e784e42a791fb9abc274d79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd70d25bf69314820b8aaecb65576978a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:17:49.305229 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab4dcd2-eb1b-495d-b601-b36cdea2514c service cinder] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:17:49.305725 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab4dcd2-eb1b-495d-b601-b36cdea2514c service cinder] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:17:49.306015 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab4dcd2-eb1b-495d-b601-b36cdea2514c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:17:49.430030 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab4dcd2-eb1b-495d-b601-b36cdea2514c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:17:49.431600 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab4dcd2-eb1b-495d-b601-b36cdea2514c service cinder] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:17:49.468644 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 825/1626] 10.222.0.23 () {60 vars in 1365 bytes} [Thu May 19 12:17:49 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:17:49.469907 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab4dcd2-eb1b-495d-b601-b36cdea2514c service cinder] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:17:49.479574 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 802/1627] 10.222.0.23 () {60 vars in 1152 bytes} [Thu May 19 12:17:49 2022] POST /identity/v3/projects => generated 373 bytes in 349 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 12:17:49.485548 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccda7ca8-cf38-4fa7-afb2-87c387f2ad58 admin admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:17:49.501466 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab4dcd2-eb1b-495d-b601-b36cdea2514c service cinder] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:17:49.631949 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:17:49.681009 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4597c960-d8c3-4c70-b7be-6702669a9f08 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8800254c0e784e42a791fb9abc274d79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd70d25bf69314820b8aaecb65576978a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:17:49.683703 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4597c960-d8c3-4c70-b7be-6702669a9f08 service cinder] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:17:49.685807 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4597c960-d8c3-4c70-b7be-6702669a9f08 service cinder] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:17:49.686768 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4597c960-d8c3-4c70-b7be-6702669a9f08 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:17:49.707506 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccda7ca8-cf38-4fa7-afb2-87c387f2ad58 admin admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:17:49.722809 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681bd7df-ce0e-4681-8e31-2bb6edb7a4b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:17:49.723654 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681bd7df-ce0e-4681-8e31-2bb6edb7a4b3 admin admin] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:17:49.723957 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681bd7df-ce0e-4681-8e31-2bb6edb7a4b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:17:49.724239 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681bd7df-ce0e-4681-8e31-2bb6edb7a4b3 admin admin] PATH_INFO: `/v3/users` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:17:49.725205 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681bd7df-ce0e-4681-8e31-2bb6edb7a4b3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:17:49.728409 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681bd7df-ce0e-4681-8e31-2bb6edb7a4b3 admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:17:49.753278 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:17:49.836659 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4597c960-d8c3-4c70-b7be-6702669a9f08 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:17:49.841671 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4597c960-d8c3-4c70-b7be-6702669a9f08 service cinder] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:17:49.873996 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 826/1628] 10.222.0.23 () {60 vars in 1365 bytes} [Thu May 19 12:17:49 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:17:49.895087 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 803/1629] 10.222.0.23 () {60 vars in 1143 bytes} [Thu May 19 12:17:49 2022] POST /identity/v3/users => generated 293 bytes in 407 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 12:17:49.896828 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4597c960-d8c3-4c70-b7be-6702669a9f08 service cinder] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:17:49.938656 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4597c960-d8c3-4c70-b7be-6702669a9f08 service cinder] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:17:49.973630 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eec5e2d-0ebf-4da5-8043-a39f2665b3b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:17:49.973630 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eec5e2d-0ebf-4da5-8043-a39f2665b3b9 admin admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:17:49.973630 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eec5e2d-0ebf-4da5-8043-a39f2665b3b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:17:49.973630 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eec5e2d-0ebf-4da5-8043-a39f2665b3b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:17:49.976690 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eec5e2d-0ebf-4da5-8043-a39f2665b3b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:17:49.983484 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eec5e2d-0ebf-4da5-8043-a39f2665b3b9 admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:17:50.061879 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 827/1630] 10.222.0.23 () {58 vars in 1121 bytes} [Thu May 19 12:17:49 2022] GET /identity/v3/roles => generated 1420 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 12:17:50.064922 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681bd7df-ce0e-4681-8e31-2bb6edb7a4b3 admin admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:17:50.098353 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681bd7df-ce0e-4681-8e31-2bb6edb7a4b3 admin admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:17:50.118287 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c6d411-5bac-46f4-9639-1ddafa5722c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:17:50.119738 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c6d411-5bac-46f4-9639-1ddafa5722c9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:17:50.120196 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c6d411-5bac-46f4-9639-1ddafa5722c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:17:50.120637 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c6d411-5bac-46f4-9639-1ddafa5722c9 admin admin] PATH_INFO: `/v3/projects/96b07d3cf30e4789947e5a79b440d720/users/8862cd4c12ac45fe9f158427515947c3/roles/9db5671c65d3414ea2682afa2ccf1403` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:17:50.145993 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eec5e2d-0ebf-4da5-8043-a39f2665b3b9 admin admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:17:50.171229 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eec5e2d-0ebf-4da5-8043-a39f2665b3b9 admin admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:17:50.176803 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c6d411-5bac-46f4-9639-1ddafa5722c9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=96b07d3cf30e4789947e5a79b440d720, user_id=8862cd4c12ac45fe9f158427515947c3, role_id=9db5671c65d3414ea2682afa2ccf1403)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:17:50.184248 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c6d411-5bac-46f4-9639-1ddafa5722c9 admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:17:50.186332 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f183eb-7fb4-4399-95a8-d67554a996d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:17:50.187288 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f183eb-7fb4-4399-95a8-d67554a996d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:17:50.187590 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f183eb-7fb4-4399-95a8-d67554a996d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:17:50.187902 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f183eb-7fb4-4399-95a8-d67554a996d4 admin admin] PATH_INFO: `/v3/users/d95e4c71778448eb96109cf89365522a` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:17:50.189599 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f183eb-7fb4-4399-95a8-d67554a996d4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d95e4c71778448eb96109cf89365522a)` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:17:50.190671 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: WARNING py.warnings [None req-88f183eb-7fb4-4399-95a8-d67554a996d4 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 19 12:17:50.190671 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: warnings.warn(msg) May 19 12:17:50.190671 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: May 19 12:17:50.192011 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f183eb-7fb4-4399-95a8-d67554a996d4 admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:17:50.314306 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 804/1631] 10.222.0.23 () {60 vars in 1482 bytes} [Thu May 19 12:17:50 2022] PUT /identity/v3/projects/96b07d3cf30e4789947e5a79b440d720/users/8862cd4c12ac45fe9f158427515947c3/roles/9db5671c65d3414ea2682afa2ccf1403 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:17:50.324700 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c6d411-5bac-46f4-9639-1ddafa5722c9 admin admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:17:50.598746 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c6d411-5bac-46f4-9639-1ddafa5722c9 admin admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:17:50.605787 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf75aa5-387d-43d3-88b9-98177ce46d04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:17:50.606874 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf75aa5-387d-43d3-88b9-98177ce46d04 admin admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:17:50.607353 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf75aa5-387d-43d3-88b9-98177ce46d04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:17:50.607825 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf75aa5-387d-43d3-88b9-98177ce46d04 admin admin] PATH_INFO: `/v3/roles` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:17:50.626718 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf75aa5-387d-43d3-88b9-98177ce46d04 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:17:50.626718 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf75aa5-387d-43d3-88b9-98177ce46d04 admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:17:50.627379 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 805/1632] 10.222.0.23 () {58 vars in 1121 bytes} [Thu May 19 12:17:50 2022] GET /identity/v3/roles => generated 1420 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 12:17:50.629807 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6126fa1-b299-41f9-9329-c38889f64f95 None None] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:17:50.630404 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6126fa1-b299-41f9-9329-c38889f64f95 None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:17:50.631184 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6126fa1-b299-41f9-9329-c38889f64f95 None None] PATH_INFO: `/` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:17:50.634534 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 806/1633] 10.222.0.23 () {54 vars in 961 bytes} [Thu May 19 12:17:50 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 12:17:50.635716 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6126fa1-b299-41f9-9329-c38889f64f95 None None] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:17:50.661787 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6126fa1-b299-41f9-9329-c38889f64f95 None None] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:17:50.668308 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe08a0d4-e07f-4564-b2f1-334709196fac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:17:50.670279 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe08a0d4-e07f-4564-b2f1-334709196fac admin admin] REQUEST_METHOD: `PUT` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:17:50.670875 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe08a0d4-e07f-4564-b2f1-334709196fac admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:17:50.671234 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe08a0d4-e07f-4564-b2f1-334709196fac admin admin] PATH_INFO: `/v3/projects/96b07d3cf30e4789947e5a79b440d720/users/8862cd4c12ac45fe9f158427515947c3/roles/9db5671c65d3414ea2682afa2ccf1403` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:17:50.673924 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe08a0d4-e07f-4564-b2f1-334709196fac admin admin] RBAC: Authorizing `identity:create_grant(project_id=96b07d3cf30e4789947e5a79b440d720, user_id=8862cd4c12ac45fe9f158427515947c3, role_id=9db5671c65d3414ea2682afa2ccf1403)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:17:50.677327 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe08a0d4-e07f-4564-b2f1-334709196fac admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:17:50.697064 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 807/1634] 10.222.0.23 () {60 vars in 1482 bytes} [Thu May 19 12:17:50 2022] PUT /identity/v3/projects/96b07d3cf30e4789947e5a79b440d720/users/8862cd4c12ac45fe9f158427515947c3/roles/9db5671c65d3414ea2682afa2ccf1403 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:17:50.698489 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc78318a-73a2-4c76-8f2b-5764681aa5b8 None None] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:17:50.698865 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc78318a-73a2-4c76-8f2b-5764681aa5b8 None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:17:50.699140 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc78318a-73a2-4c76-8f2b-5764681aa5b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:17:50.746443 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.auth.core [None req-bc78318a-73a2-4c76-8f2b-5764681aa5b8 None None] MFA Rules not processed for user `34a98d75d84b4c67981377e0f26eca62`. Rule list: `[]` (Enabled: `True`). {{(pid=80539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:17:50.749903 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-bc78318a-73a2-4c76-8f2b-5764681aa5b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:17:50.764618 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-88f183eb-7fb4-4399-95a8-d67554a996d4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd95e4c71778448eb96109cf89365522a'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:17:50.782231 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:17:50.799765 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-88f183eb-7fb4-4399-95a8-d67554a996d4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd95e4c71778448eb96109cf89365522a'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:17:50.800787 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-88f183eb-7fb4-4399-95a8-d67554a996d4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd95e4c71778448eb96109cf89365522a'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:17:50.908951 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-88f183eb-7fb4-4399-95a8-d67554a996d4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd95e4c71778448eb96109cf89365522a'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:17:50.971846 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-88f183eb-7fb4-4399-95a8-d67554a996d4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd95e4c71778448eb96109cf89365522a'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:17:51.005145 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 828/1635] 10.222.0.23 () {58 vars in 1223 bytes} [Thu May 19 12:17:50 2022] DELETE /identity/v3/users/d95e4c71778448eb96109cf89365522a => generated 0 bytes in 855 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 12:17:51.005369 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 808/1636] 10.222.0.23 () {58 vars in 1058 bytes} [Thu May 19 12:17:50 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 307 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:17:51.006155 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7727643c-1f92-4e1f-9679-bd0d5ae08df5 None None] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:17:51.006339 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e6c633-5eaf-48f1-93b0-aefa97f64d70 None None] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:17:51.006476 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7727643c-1f92-4e1f-9679-bd0d5ae08df5 None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:17:51.006718 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7727643c-1f92-4e1f-9679-bd0d5ae08df5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:17:51.009067 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e6c633-5eaf-48f1-93b0-aefa97f64d70 None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:17:51.009568 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e6c633-5eaf-48f1-93b0-aefa97f64d70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:17:51.072086 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.auth.core [None req-7727643c-1f92-4e1f-9679-bd0d5ae08df5 None None] MFA Rules not processed for user `9e66a142d53340bb8bc4a0a9a9f48d59`. Rule list: `[]` (Enabled: `True`). {{(pid=80539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:17:51.073347 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-7727643c-1f92-4e1f-9679-bd0d5ae08df5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:17:51.130137 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.auth.core [None req-93e6c633-5eaf-48f1-93b0-aefa97f64d70 None None] MFA Rules not processed for user `8862cd4c12ac45fe9f158427515947c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:17:51.133007 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-93e6c633-5eaf-48f1-93b0-aefa97f64d70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:17:51.169522 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:17:51.236329 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 809/1637] 10.222.0.23 () {58 vars in 957 bytes} [Thu May 19 12:17:51 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:17:51.236329 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3589082-1590-419b-83f3-f4237bbece6f None None] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:17:51.236329 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3589082-1590-419b-83f3-f4237bbece6f None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:17:51.236329 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3589082-1590-419b-83f3-f4237bbece6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:17:51.278509 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.auth.core [None req-b3589082-1590-419b-83f3-f4237bbece6f None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:17:51.282310 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-b3589082-1590-419b-83f3-f4237bbece6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:17:51.323768 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 829/1638] 10.222.0.23 () {58 vars in 957 bytes} [Thu May 19 12:17:51 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 322 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:17:51.342676 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1df1a6-2555-4e52-b70f-6e78a4a2d0ce None None] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:17:51.342970 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1df1a6-2555-4e52-b70f-6e78a4a2d0ce None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:17:51.343253 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1df1a6-2555-4e52-b70f-6e78a4a2d0ce None None] PATH_INFO: `/` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:17:51.347711 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 830/1639] 10.222.0.23 () {54 vars in 912 bytes} [Thu May 19 12:17:51 2022] GET /identity => generated 270 bytes in 23 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 12:17:51.358365 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863fb9e7-3415-4df0-96dc-68aa5378688e None None] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:17:51.358768 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863fb9e7-3415-4df0-96dc-68aa5378688e None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:17:51.359054 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863fb9e7-3415-4df0-96dc-68aa5378688e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:17:51.397749 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.auth.core [None req-863fb9e7-3415-4df0-96dc-68aa5378688e None None] MFA Rules not processed for user `9e66a142d53340bb8bc4a0a9a9f48d59`. Rule list: `[]` (Enabled: `True`). {{(pid=80540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:17:51.401353 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 810/1640] 10.222.0.23 () {58 vars in 957 bytes} [Thu May 19 12:17:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:17:51.401545 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-863fb9e7-3415-4df0-96dc-68aa5378688e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:17:51.401617 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3589082-1590-419b-83f3-f4237bbece6f None None] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:17:51.404395 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-b3589082-1590-419b-83f3-f4237bbece6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:17:51.450178 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 831/1641] 10.222.0.23 () {58 vars in 957 bytes} [Thu May 19 12:17:51 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:17:51.453531 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835c53cf-46d9-4447-857b-52e1a45e37e0 None None] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:17:51.453829 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835c53cf-46d9-4447-857b-52e1a45e37e0 None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:17:51.456526 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835c53cf-46d9-4447-857b-52e1a45e37e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:17:51.486107 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3589082-1590-419b-83f3-f4237bbece6f None None] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:17:51.490477 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6253efc7-0632-40fa-9b68-071e11900a5c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8800254c0e784e42a791fb9abc274d79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd70d25bf69314820b8aaecb65576978a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:17:51.490703 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.auth.core [None req-835c53cf-46d9-4447-857b-52e1a45e37e0 None None] MFA Rules not processed for user `34a98d75d84b4c67981377e0f26eca62`. Rule list: `[]` (Enabled: `True`). {{(pid=80540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:17:51.492144 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6253efc7-0632-40fa-9b68-071e11900a5c service cinder] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:17:51.492489 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-835c53cf-46d9-4447-857b-52e1a45e37e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:17:51.492669 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6253efc7-0632-40fa-9b68-071e11900a5c service cinder] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:17:51.493078 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6253efc7-0632-40fa-9b68-071e11900a5c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:17:51.508661 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6253efc7-0632-40fa-9b68-071e11900a5c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:17:51.511691 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6253efc7-0632-40fa-9b68-071e11900a5c service cinder] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:17:51.525935 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 811/1642] 10.222.0.23 () {60 vars in 1365 bytes} [Thu May 19 12:17:51 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:17:51.527194 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6253efc7-0632-40fa-9b68-071e11900a5c service cinder] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:17:51.528943 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-6253efc7-0632-40fa-9b68-071e11900a5c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:17:51.647031 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6253efc7-0632-40fa-9b68-071e11900a5c service cinder] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:17:51.663548 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f67069c-ac75-4137-8548-c0d1d429875e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a00b3fa7497456895e5e0bf105c4fc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd70d25bf69314820b8aaecb65576978a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:17:51.663765 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f67069c-ac75-4137-8548-c0d1d429875e service neutron] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:17:51.664098 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f67069c-ac75-4137-8548-c0d1d429875e service neutron] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:17:51.664470 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f67069c-ac75-4137-8548-c0d1d429875e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:17:51.679669 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f67069c-ac75-4137-8548-c0d1d429875e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:17:51.686972 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f67069c-ac75-4137-8548-c0d1d429875e service neutron] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:17:51.714253 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 832/1643] 10.222.0.23 () {58 vars in 1009 bytes} [Thu May 19 12:17:51 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 263 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:17:51.716200 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 812/1644] 10.222.0.23 () {60 vars in 1365 bytes} [Thu May 19 12:17:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:17:51.716325 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835c53cf-46d9-4447-857b-52e1a45e37e0 None None] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:17:51.717890 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-835c53cf-46d9-4447-857b-52e1a45e37e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:17:51.737418 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835c53cf-46d9-4447-857b-52e1a45e37e0 None None] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:17:51.753658 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29864765-5cfe-48c2-bd9a-dc5bbe3fc320 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34a98d75d84b4c67981377e0f26eca62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd70d25bf69314820b8aaecb65576978a', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:17:51.757962 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29864765-5cfe-48c2-bd9a-dc5bbe3fc320 service nova] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:17:51.757962 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29864765-5cfe-48c2-bd9a-dc5bbe3fc320 service nova] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:17:51.758702 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29864765-5cfe-48c2-bd9a-dc5bbe3fc320 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:17:51.777024 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f67069c-ac75-4137-8548-c0d1d429875e service neutron] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:17:51.782578 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-8f67069c-ac75-4137-8548-c0d1d429875e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:17:51.791540 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29864765-5cfe-48c2-bd9a-dc5bbe3fc320 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:17:51.796759 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29864765-5cfe-48c2-bd9a-dc5bbe3fc320 service nova] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:17:51.797104 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f67069c-ac75-4137-8548-c0d1d429875e service neutron] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:17:51.802468 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb78b5c-2fee-4970-b2b4-cf399ecba65c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8800254c0e784e42a791fb9abc274d79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd70d25bf69314820b8aaecb65576978a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:17:51.804393 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb78b5c-2fee-4970-b2b4-cf399ecba65c service cinder] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:17:51.804393 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb78b5c-2fee-4970-b2b4-cf399ecba65c service cinder] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:17:51.804393 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb78b5c-2fee-4970-b2b4-cf399ecba65c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:17:51.827183 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb78b5c-2fee-4970-b2b4-cf399ecba65c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:17:51.830669 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb78b5c-2fee-4970-b2b4-cf399ecba65c service cinder] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:17:51.834466 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 833/1645] 10.222.0.23 () {60 vars in 1365 bytes} [Thu May 19 12:17:51 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:17:51.887032 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 813/1646] 10.222.0.23 () {60 vars in 1365 bytes} [Thu May 19 12:17:51 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 12:17:52.107225 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65106f4-7bbe-42d7-9700-4efc59fc1d47 None None] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:17:52.107563 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65106f4-7bbe-42d7-9700-4efc59fc1d47 None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:17:52.107860 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65106f4-7bbe-42d7-9700-4efc59fc1d47 None None] PATH_INFO: `/` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:17:52.110994 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 834/1647] 10.222.0.23 () {54 vars in 912 bytes} [Thu May 19 12:17:52 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 12:17:52.125477 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9fa814-e7d1-4336-9507-1ffd18c985ed None None] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:17:52.125806 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9fa814-e7d1-4336-9507-1ffd18c985ed None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:17:52.126148 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9fa814-e7d1-4336-9507-1ffd18c985ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:17:52.137662 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:17:52.172958 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.auth.core [None req-1b9fa814-e7d1-4336-9507-1ffd18c985ed None None] MFA Rules not processed for user `34a98d75d84b4c67981377e0f26eca62`. Rule list: `[]` (Enabled: `True`). {{(pid=80539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:17:52.174272 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-1b9fa814-e7d1-4336-9507-1ffd18c985ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:17:52.188520 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65106f4-7bbe-42d7-9700-4efc59fc1d47 None None] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:17:52.191260 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-b65106f4-7bbe-42d7-9700-4efc59fc1d47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:17:52.198497 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 814/1648] 10.222.0.23 () {58 vars in 1009 bytes} [Thu May 19 12:17:52 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:17:52.222327 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9fa814-e7d1-4336-9507-1ffd18c985ed None None] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:17:52.223969 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-1b9fa814-e7d1-4336-9507-1ffd18c985ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:17:52.230686 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65106f4-7bbe-42d7-9700-4efc59fc1d47 None None] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:17:52.239178 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89100c2-94cb-4696-b88d-9241084ec00b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:17:52.246038 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89100c2-94cb-4696-b88d-9241084ec00b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:17:52.246038 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89100c2-94cb-4696-b88d-9241084ec00b admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:17:52.246038 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89100c2-94cb-4696-b88d-9241084ec00b admin admin] PATH_INFO: `/v3/users/ec3d521695a64c62a6a2844594ac420e` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:17:52.246038 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89100c2-94cb-4696-b88d-9241084ec00b admin admin] RBAC: Authorizing `identity:delete_user(user_id=ec3d521695a64c62a6a2844594ac420e)` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:17:52.264915 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89100c2-94cb-4696-b88d-9241084ec00b admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:17:52.360886 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:DELETE=6,keystone:SELECT=93,keystone:INSERT=1 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:17:52.377202 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9fa814-e7d1-4336-9507-1ffd18c985ed None None] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:17:52.381607 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5940cd-12a6-4dfb-9afd-fdf6546bcc3c service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '21697802f2be42f9baa8e07490d5ebdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd70d25bf69314820b8aaecb65576978a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:17:52.382861 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5940cd-12a6-4dfb-9afd-fdf6546bcc3c service glance] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:17:52.383151 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5940cd-12a6-4dfb-9afd-fdf6546bcc3c service glance] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:17:52.383429 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5940cd-12a6-4dfb-9afd-fdf6546bcc3c service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:17:52.412264 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5940cd-12a6-4dfb-9afd-fdf6546bcc3c service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:17:52.414604 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5940cd-12a6-4dfb-9afd-fdf6546bcc3c service glance] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:17:52.438177 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 815/1649] 10.222.0.23 () {60 vars in 1365 bytes} [Thu May 19 12:17:52 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 217 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 12:17:52.634433 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-a89100c2-94cb-4696-b88d-9241084ec00b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ec3d521695a64c62a6a2844594ac420e'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:17:52.676017 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-a89100c2-94cb-4696-b88d-9241084ec00b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ec3d521695a64c62a6a2844594ac420e'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:17:52.677986 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-a89100c2-94cb-4696-b88d-9241084ec00b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ec3d521695a64c62a6a2844594ac420e'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:17:52.761766 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-a89100c2-94cb-4696-b88d-9241084ec00b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ec3d521695a64c62a6a2844594ac420e'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:17:52.790789 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-a89100c2-94cb-4696-b88d-9241084ec00b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ec3d521695a64c62a6a2844594ac420e'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:17:52.818149 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 835/1650] 10.222.0.23 () {58 vars in 1223 bytes} [Thu May 19 12:17:52 2022] DELETE /identity/v3/users/ec3d521695a64c62a6a2844594ac420e => generated 0 bytes in 627 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:17:52.827222 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec14d00-3537-477f-8961-c85b5f7417d1 None None] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:17:52.827972 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec14d00-3537-477f-8961-c85b5f7417d1 None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:17:52.828657 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec14d00-3537-477f-8961-c85b5f7417d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:17:52.896383 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.auth.core [None req-cec14d00-3537-477f-8961-c85b5f7417d1 None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:17:52.900179 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-cec14d00-3537-477f-8961-c85b5f7417d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:17:53.127056 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 816/1651] 10.222.0.23 () {58 vars in 957 bytes} [Thu May 19 12:17:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 307 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:17:53.221529 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89100c2-94cb-4696-b88d-9241084ec00b admin admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:17:53.224367 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-a89100c2-94cb-4696-b88d-9241084ec00b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:17:53.440033 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89100c2-94cb-4696-b88d-9241084ec00b admin admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:17:53.454257 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ad269d-3389-4834-bdd7-7e6fbb0075b5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a00b3fa7497456895e5e0bf105c4fc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd70d25bf69314820b8aaecb65576978a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:17:53.454961 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ad269d-3389-4834-bdd7-7e6fbb0075b5 service neutron] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:17:53.455256 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ad269d-3389-4834-bdd7-7e6fbb0075b5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:17:53.455540 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ad269d-3389-4834-bdd7-7e6fbb0075b5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:17:53.484652 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ad269d-3389-4834-bdd7-7e6fbb0075b5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:17:53.486549 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ad269d-3389-4834-bdd7-7e6fbb0075b5 service neutron] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:17:53.486964 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec14d00-3537-477f-8961-c85b5f7417d1 None None] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:17:53.490187 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-cec14d00-3537-477f-8961-c85b5f7417d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:17:53.513060 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 836/1652] 10.222.0.23 () {60 vars in 1365 bytes} [Thu May 19 12:17:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:17:53.513722 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec14d00-3537-477f-8961-c85b5f7417d1 None None] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:17:53.522405 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52484502-5135-4003-b695-a12143f48758 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:17:53.525050 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52484502-5135-4003-b695-a12143f48758 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:17:53.525902 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52484502-5135-4003-b695-a12143f48758 admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:17:53.526776 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52484502-5135-4003-b695-a12143f48758 admin admin] PATH_INFO: `/v3/users/08994739116f4675b5a9e4e11c42ce5c` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:17:53.529732 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52484502-5135-4003-b695-a12143f48758 admin admin] RBAC: Authorizing `identity:delete_user(user_id=08994739116f4675b5a9e4e11c42ce5c)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:17:53.533990 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52484502-5135-4003-b695-a12143f48758 admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:17:53.796434 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-52484502-5135-4003-b695-a12143f48758 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '08994739116f4675b5a9e4e11c42ce5c'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:17:53.827222 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-52484502-5135-4003-b695-a12143f48758 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '08994739116f4675b5a9e4e11c42ce5c'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:17:53.828979 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-52484502-5135-4003-b695-a12143f48758 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '08994739116f4675b5a9e4e11c42ce5c'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:17:53.933825 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ad269d-3389-4834-bdd7-7e6fbb0075b5 service neutron] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:17:53.935855 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-52484502-5135-4003-b695-a12143f48758 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '08994739116f4675b5a9e4e11c42ce5c'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:17:53.945170 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-76ad269d-3389-4834-bdd7-7e6fbb0075b5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:17:53.958642 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-52484502-5135-4003-b695-a12143f48758 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '08994739116f4675b5a9e4e11c42ce5c'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:17:53.984461 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 817/1653] 10.222.0.23 () {58 vars in 1223 bytes} [Thu May 19 12:17:53 2022] DELETE /identity/v3/users/08994739116f4675b5a9e4e11c42ce5c => generated 0 bytes in 499 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:17:53.993092 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e35e74-196b-4474-9821-25cb94b174fb None None] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:17:53.993689 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e35e74-196b-4474-9821-25cb94b174fb None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:17:53.993915 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e35e74-196b-4474-9821-25cb94b174fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:17:54.010029 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ad269d-3389-4834-bdd7-7e6fbb0075b5 service neutron] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:17:54.057860 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.auth.core [None req-62e35e74-196b-4474-9821-25cb94b174fb None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:17:54.059112 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-62e35e74-196b-4474-9821-25cb94b174fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:17:54.102584 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:17:54.171447 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3198778-b689-4c2b-a344-69946001785c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:17:54.172243 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3198778-b689-4c2b-a344-69946001785c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:17:54.172745 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3198778-b689-4c2b-a344-69946001785c admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:17:54.173249 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3198778-b689-4c2b-a344-69946001785c admin admin] PATH_INFO: `/v3/projects/482e9ea9fe48482fa644191c2956b4ec` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:17:54.175714 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3198778-b689-4c2b-a344-69946001785c admin admin] RBAC: Authorizing `identity:delete_project(project_id=482e9ea9fe48482fa644191c2956b4ec)` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:17:54.179291 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3198778-b689-4c2b-a344-69946001785c admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:17:54.281613 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 818/1654] 10.222.0.23 () {58 vars in 957 bytes} [Thu May 19 12:17:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 293 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:17:54.287422 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c8fb80-4bc7-4f59-ae03-156c4841f6af None None] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:17:54.288007 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c8fb80-4bc7-4f59-ae03-156c4841f6af None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:17:54.288636 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c8fb80-4bc7-4f59-ae03-156c4841f6af None None] PATH_INFO: `/` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:17:54.290799 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 819/1655] 10.222.0.23 () {54 vars in 912 bytes} [Thu May 19 12:17:54 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 12:17:54.296615 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23049446-3acf-40b5-954d-2f74549f5071 None None] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:17:54.297208 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23049446-3acf-40b5-954d-2f74549f5071 None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:17:54.297682 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23049446-3acf-40b5-954d-2f74549f5071 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:17:54.364177 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.auth.core [None req-23049446-3acf-40b5-954d-2f74549f5071 None None] MFA Rules not processed for user `1a00b3fa7497456895e5e0bf105c4fc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:17:54.365946 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-23049446-3acf-40b5-954d-2f74549f5071 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:17:54.386936 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:17:54.411144 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-b3198778-b689-4c2b-a344-69946001785c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '482e9ea9fe48482fa644191c2956b4ec', 'request_id': 'req-b3198778-b689-4c2b-a344-69946001785c'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:17:54.441576 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-b3198778-b689-4c2b-a344-69946001785c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '482e9ea9fe48482fa644191c2956b4ec', 'request_id': 'req-b3198778-b689-4c2b-a344-69946001785c'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:17:54.459515 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-b3198778-b689-4c2b-a344-69946001785c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '482e9ea9fe48482fa644191c2956b4ec', 'request_id': 'req-b3198778-b689-4c2b-a344-69946001785c'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:17:54.488421 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 820/1656] 10.222.0.23 () {58 vars in 1009 bytes} [Thu May 19 12:17:54 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 12:17:54.490305 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23049446-3acf-40b5-954d-2f74549f5071 None None] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:17:54.493599 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-23049446-3acf-40b5-954d-2f74549f5071 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:17:54.513935 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23049446-3acf-40b5-954d-2f74549f5071 None None] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:17:54.520669 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43911a08-ed33-40ad-a21c-1dd1ab77c432 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a00b3fa7497456895e5e0bf105c4fc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd70d25bf69314820b8aaecb65576978a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:17:54.521694 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43911a08-ed33-40ad-a21c-1dd1ab77c432 service neutron] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:17:54.522560 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43911a08-ed33-40ad-a21c-1dd1ab77c432 service neutron] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:17:54.523942 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43911a08-ed33-40ad-a21c-1dd1ab77c432 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:17:54.543243 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-b3198778-b689-4c2b-a344-69946001785c admin admin] The token cache is being invalidate because project 482e9ea9fe48482fa644191c2956b4ec was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80540) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 12:17:54.543691 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-b3198778-b689-4c2b-a344-69946001785c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:17:54.544436 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-b3198778-b689-4c2b-a344-69946001785c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:17:54.550186 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 837/1657] 10.222.0.23 () {58 vars in 1232 bytes} [Thu May 19 12:17:53 2022] DELETE /identity/v3/projects/482e9ea9fe48482fa644191c2956b4ec => generated 0 bytes in 616 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:17:54.554238 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3198778-b689-4c2b-a344-69946001785c admin admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:17:54.556149 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-b3198778-b689-4c2b-a344-69946001785c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:17:54.680231 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43911a08-ed33-40ad-a21c-1dd1ab77c432 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:17:54.684232 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43911a08-ed33-40ad-a21c-1dd1ab77c432 service neutron] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:17:54.688273 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3198778-b689-4c2b-a344-69946001785c admin admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:17:54.694413 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ba6825-fb2f-4e16-8299-4c3148717890 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:17:54.695608 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ba6825-fb2f-4e16-8299-4c3148717890 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:17:54.696750 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ba6825-fb2f-4e16-8299-4c3148717890 admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:17:54.697261 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ba6825-fb2f-4e16-8299-4c3148717890 admin admin] PATH_INFO: `/v3/users/01aa7be0f56246be8ce9c1aa72567851` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:17:54.699093 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ba6825-fb2f-4e16-8299-4c3148717890 admin admin] RBAC: Authorizing `identity:delete_user(user_id=01aa7be0f56246be8ce9c1aa72567851)` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:17:54.701548 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ba6825-fb2f-4e16-8299-4c3148717890 admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:17:54.719248 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 821/1658] 10.222.0.23 () {60 vars in 1365 bytes} [Thu May 19 12:17:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 230 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:17:54.720956 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43911a08-ed33-40ad-a21c-1dd1ab77c432 service neutron] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:17:54.724623 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-43911a08-ed33-40ad-a21c-1dd1ab77c432 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:17:54.752529 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43911a08-ed33-40ad-a21c-1dd1ab77c432 service neutron] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:17:54.760543 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6403d0e8-553f-44e9-b64f-52fb46c8a43a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a00b3fa7497456895e5e0bf105c4fc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd70d25bf69314820b8aaecb65576978a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:17:54.761652 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6403d0e8-553f-44e9-b64f-52fb46c8a43a service neutron] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:17:54.762170 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6403d0e8-553f-44e9-b64f-52fb46c8a43a service neutron] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:17:54.762695 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6403d0e8-553f-44e9-b64f-52fb46c8a43a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:17:54.767320 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-6403d0e8-553f-44e9-b64f-52fb46c8a43a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:17:54.804126 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6403d0e8-553f-44e9-b64f-52fb46c8a43a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:17:54.804126 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6403d0e8-553f-44e9-b64f-52fb46c8a43a service neutron] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:17:54.820400 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 822/1659] 10.222.0.23 () {60 vars in 1365 bytes} [Thu May 19 12:17:54 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:17:55.022423 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-96ba6825-fb2f-4e16-8299-4c3148717890 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '01aa7be0f56246be8ce9c1aa72567851'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:17:55.081428 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-96ba6825-fb2f-4e16-8299-4c3148717890 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '01aa7be0f56246be8ce9c1aa72567851'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:17:55.082959 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-96ba6825-fb2f-4e16-8299-4c3148717890 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '01aa7be0f56246be8ce9c1aa72567851'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:17:55.163259 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-96ba6825-fb2f-4e16-8299-4c3148717890 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '01aa7be0f56246be8ce9c1aa72567851'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:17:55.192451 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-96ba6825-fb2f-4e16-8299-4c3148717890 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '01aa7be0f56246be8ce9c1aa72567851'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:17:55.215151 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 838/1660] 10.222.0.23 () {58 vars in 1223 bytes} [Thu May 19 12:17:54 2022] DELETE /identity/v3/users/01aa7be0f56246be8ce9c1aa72567851 => generated 0 bytes in 661 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:17:55.271624 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6403d0e8-553f-44e9-b64f-52fb46c8a43a service neutron] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:17:55.274280 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-6403d0e8-553f-44e9-b64f-52fb46c8a43a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:17:55.295690 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6403d0e8-553f-44e9-b64f-52fb46c8a43a service neutron] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:17:55.345760 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:17:55.495865 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65496dd8-2e7c-44ad-89a6-42a2d69281d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:17:55.497238 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65496dd8-2e7c-44ad-89a6-42a2d69281d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:17:55.497542 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65496dd8-2e7c-44ad-89a6-42a2d69281d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:17:55.501713 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65496dd8-2e7c-44ad-89a6-42a2d69281d3 admin admin] PATH_INFO: `/v3/projects/5a530cf15e11460b898957828f7d858e` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:17:55.501835 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65496dd8-2e7c-44ad-89a6-42a2d69281d3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5a530cf15e11460b898957828f7d858e)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:17:55.505053 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: WARNING py.warnings [None req-65496dd8-2e7c-44ad-89a6-42a2d69281d3 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 19 12:17:55.505053 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: warnings.warn(msg) May 19 12:17:55.505053 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: May 19 12:17:55.509495 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65496dd8-2e7c-44ad-89a6-42a2d69281d3 admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:17:55.781427 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-65496dd8-2e7c-44ad-89a6-42a2d69281d3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5a530cf15e11460b898957828f7d858e', 'request_id': 'req-65496dd8-2e7c-44ad-89a6-42a2d69281d3'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:17:55.833199 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-65496dd8-2e7c-44ad-89a6-42a2d69281d3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5a530cf15e11460b898957828f7d858e', 'request_id': 'req-65496dd8-2e7c-44ad-89a6-42a2d69281d3'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:17:55.940876 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-65496dd8-2e7c-44ad-89a6-42a2d69281d3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5a530cf15e11460b898957828f7d858e', 'request_id': 'req-65496dd8-2e7c-44ad-89a6-42a2d69281d3'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:17:55.968683 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-65496dd8-2e7c-44ad-89a6-42a2d69281d3 admin admin] The token cache is being invalidate because project 5a530cf15e11460b898957828f7d858e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80539) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 12:17:55.968977 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-65496dd8-2e7c-44ad-89a6-42a2d69281d3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:17:55.973909 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-65496dd8-2e7c-44ad-89a6-42a2d69281d3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:17:55.977025 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 823/1661] 10.222.0.23 () {58 vars in 1232 bytes} [Thu May 19 12:17:55 2022] DELETE /identity/v3/projects/5a530cf15e11460b898957828f7d858e => generated 0 bytes in 705 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 12:17:55.985127 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb78563e-ca69-4f54-b6cc-7bf58c6a620a None None] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:17:55.985762 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb78563e-ca69-4f54-b6cc-7bf58c6a620a None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:17:55.986064 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb78563e-ca69-4f54-b6cc-7bf58c6a620a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:17:56.065937 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.auth.core [None req-cb78563e-ca69-4f54-b6cc-7bf58c6a620a None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:17:56.071165 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-cb78563e-ca69-4f54-b6cc-7bf58c6a620a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:17:56.242626 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:17:56.302044 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 839/1662] 10.222.0.23 () {58 vars in 957 bytes} [Thu May 19 12:17:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 319 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:17:56.310458 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813e8185-c107-445e-a464-8b7c8eca6f4c None None] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:17:56.310828 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813e8185-c107-445e-a464-8b7c8eca6f4c None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:17:56.311970 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813e8185-c107-445e-a464-8b7c8eca6f4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:17:56.382187 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.auth.core [None req-813e8185-c107-445e-a464-8b7c8eca6f4c None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:17:56.395684 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-813e8185-c107-445e-a464-8b7c8eca6f4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:17:56.449161 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 824/1663] 10.222.0.23 () {58 vars in 957 bytes} [Thu May 19 12:17:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:17:56.466301 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb78563e-ca69-4f54-b6cc-7bf58c6a620a None None] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:17:56.565006 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb78563e-ca69-4f54-b6cc-7bf58c6a620a None None] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:17:56.601718 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afefb14b-3904-4167-b21e-dbe3c7778f9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:17:56.605012 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afefb14b-3904-4167-b21e-dbe3c7778f9c admin admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:17:56.605543 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afefb14b-3904-4167-b21e-dbe3c7778f9c admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:17:56.606057 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afefb14b-3904-4167-b21e-dbe3c7778f9c admin admin] PATH_INFO: `/v3/domains` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:17:56.609375 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a678851-572a-4de5-b4a3-b6e983e90df7 None None] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:17:56.610405 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afefb14b-3904-4167-b21e-dbe3c7778f9c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:17:56.610957 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afefb14b-3904-4167-b21e-dbe3c7778f9c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:17:56.611380 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a678851-572a-4de5-b4a3-b6e983e90df7 None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:17:56.612200 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a678851-572a-4de5-b4a3-b6e983e90df7 None None] PATH_INFO: `/` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:17:56.615719 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 825/1664] 10.222.0.23 () {54 vars in 960 bytes} [Thu May 19 12:17:56 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 12:17:56.638733 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ba2764-a76e-4d72-b7a5-04dd0c3d8e97 None None] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:17:56.638985 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afefb14b-3904-4167-b21e-dbe3c7778f9c admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:17:56.639287 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ba2764-a76e-4d72-b7a5-04dd0c3d8e97 None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:17:56.639287 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ba2764-a76e-4d72-b7a5-04dd0c3d8e97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:17:56.715698 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.auth.core [None req-63ba2764-a76e-4d72-b7a5-04dd0c3d8e97 None None] MFA Rules not processed for user `50f937eed4f3414fa6f19cab2a84eb5f`. Rule list: `[]` (Enabled: `True`). {{(pid=80539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:17:56.717264 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-63ba2764-a76e-4d72-b7a5-04dd0c3d8e97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:17:56.753645 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 840/1665] 10.222.0.23 () {58 vars in 1152 bytes} [Thu May 19 12:17:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:17:56.756118 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afefb14b-3904-4167-b21e-dbe3c7778f9c admin admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:17:56.765891 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-afefb14b-3904-4167-b21e-dbe3c7778f9c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:17:56.823207 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afefb14b-3904-4167-b21e-dbe3c7778f9c admin admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:17:56.854163 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45dcda50-276e-4a0f-a789-a0e498a9c470 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:17:56.855504 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dcda50-276e-4a0f-a789-a0e498a9c470 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:17:56.855972 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dcda50-276e-4a0f-a789-a0e498a9c470 admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:17:56.856630 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dcda50-276e-4a0f-a789-a0e498a9c470 admin admin] PATH_INFO: `/v3/projects/5468e69ac0c843f4b31fda5934c88728` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:17:56.858765 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45dcda50-276e-4a0f-a789-a0e498a9c470 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5468e69ac0c843f4b31fda5934c88728)` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:17:56.861745 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45dcda50-276e-4a0f-a789-a0e498a9c470 admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:17:57.027650 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 826/1666] 10.222.0.23 () {58 vars in 1057 bytes} [Thu May 19 12:17:56 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 395 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:17:57.029995 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ba2764-a76e-4d72-b7a5-04dd0c3d8e97 None None] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:17:57.071296 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ba2764-a76e-4d72-b7a5-04dd0c3d8e97 None None] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:17:57.072560 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-45dcda50-276e-4a0f-a789-a0e498a9c470 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5468e69ac0c843f4b31fda5934c88728', 'request_id': 'req-45dcda50-276e-4a0f-a789-a0e498a9c470'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:17:57.089379 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-45dcda50-276e-4a0f-a789-a0e498a9c470 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5468e69ac0c843f4b31fda5934c88728', 'request_id': 'req-45dcda50-276e-4a0f-a789-a0e498a9c470'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:17:57.105903 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-45dcda50-276e-4a0f-a789-a0e498a9c470 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5468e69ac0c843f4b31fda5934c88728', 'request_id': 'req-45dcda50-276e-4a0f-a789-a0e498a9c470'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:17:57.188653 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-45dcda50-276e-4a0f-a789-a0e498a9c470 admin admin] The token cache is being invalidate because project 5468e69ac0c843f4b31fda5934c88728 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80540) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 12:17:57.188959 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-45dcda50-276e-4a0f-a789-a0e498a9c470 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:17:57.190447 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-45dcda50-276e-4a0f-a789-a0e498a9c470 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:17:57.193578 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 841/1667] 10.222.0.23 () {58 vars in 1232 bytes} [Thu May 19 12:17:56 2022] DELETE /identity/v3/projects/5468e69ac0c843f4b31fda5934c88728 => generated 0 bytes in 439 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:17:57.194794 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45dcda50-276e-4a0f-a789-a0e498a9c470 admin admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:17:57.201167 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-45dcda50-276e-4a0f-a789-a0e498a9c470 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:17:57.240342 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:17:57.266536 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e3394d-182e-4d93-b5be-fdd2ba015d1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:17:57.267234 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e3394d-182e-4d93-b5be-fdd2ba015d1b admin admin] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:17:57.267528 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e3394d-182e-4d93-b5be-fdd2ba015d1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:17:57.268006 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e3394d-182e-4d93-b5be-fdd2ba015d1b admin admin] PATH_INFO: `/v3/projects` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:17:57.268743 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e3394d-182e-4d93-b5be-fdd2ba015d1b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:17:57.270774 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e3394d-182e-4d93-b5be-fdd2ba015d1b admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:17:57.373521 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45dcda50-276e-4a0f-a789-a0e498a9c470 admin admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:17:57.382834 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0cf5cb1-38f5-43e4-97c9-f994f09f5f26 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50f937eed4f3414fa6f19cab2a84eb5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd70d25bf69314820b8aaecb65576978a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:17:57.384082 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cf5cb1-38f5-43e4-97c9-f994f09f5f26 service placement] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:17:57.384526 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cf5cb1-38f5-43e4-97c9-f994f09f5f26 service placement] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:17:57.384953 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cf5cb1-38f5-43e4-97c9-f994f09f5f26 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:17:57.390099 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-f0cf5cb1-38f5-43e4-97c9-f994f09f5f26 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:17:57.405532 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 827/1668] 10.222.0.23 () {60 vars in 1152 bytes} [Thu May 19 12:17:57 2022] POST /identity/v3/projects => generated 405 bytes in 377 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 12:17:57.407444 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a846ea-9c9a-4efc-a298-57d5b45d2cf6 None None] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:17:57.409522 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a846ea-9c9a-4efc-a298-57d5b45d2cf6 None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:17:57.411206 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a846ea-9c9a-4efc-a298-57d5b45d2cf6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:17:57.413119 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0cf5cb1-38f5-43e4-97c9-f994f09f5f26 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:17:57.415747 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0cf5cb1-38f5-43e4-97c9-f994f09f5f26 service placement] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:17:57.435579 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 842/1669] 10.222.0.23 () {60 vars in 1365 bytes} [Thu May 19 12:17:57 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 241 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:17:57.436701 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0cf5cb1-38f5-43e4-97c9-f994f09f5f26 service placement] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:17:57.440494 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-f0cf5cb1-38f5-43e4-97c9-f994f09f5f26 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:17:57.451978 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.auth.core [None req-23a846ea-9c9a-4efc-a298-57d5b45d2cf6 None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:17:57.456298 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-23a846ea-9c9a-4efc-a298-57d5b45d2cf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:17:57.473249 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0cf5cb1-38f5-43e4-97c9-f994f09f5f26 service placement] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:17:57.602027 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:17:57.660307 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 828/1670] 10.222.0.23 () {58 vars in 957 bytes} [Thu May 19 12:17:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:17:57.661919 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68811167-2d2b-4dcd-8317-88b3eede486f None None] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:17:57.662214 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68811167-2d2b-4dcd-8317-88b3eede486f None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:17:57.662494 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68811167-2d2b-4dcd-8317-88b3eede486f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:17:57.665952 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7650654d-21c8-4e80-93bd-1c71b30d7b07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:17:57.668692 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7650654d-21c8-4e80-93bd-1c71b30d7b07 admin admin] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:17:57.669163 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7650654d-21c8-4e80-93bd-1c71b30d7b07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:17:57.669592 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7650654d-21c8-4e80-93bd-1c71b30d7b07 admin admin] PATH_INFO: `/v3/users` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:17:57.670468 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7650654d-21c8-4e80-93bd-1c71b30d7b07 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:17:57.682111 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7650654d-21c8-4e80-93bd-1c71b30d7b07 admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:17:57.718983 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.auth.core [None req-68811167-2d2b-4dcd-8317-88b3eede486f None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:17:57.720259 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-68811167-2d2b-4dcd-8317-88b3eede486f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:17:57.744428 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 829/1671] 10.222.0.23 () {58 vars in 957 bytes} [Thu May 19 12:17:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:17:57.750061 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68811167-2d2b-4dcd-8317-88b3eede486f None None] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:17:57.798659 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 843/1672] 10.222.0.23 () {60 vars in 1143 bytes} [Thu May 19 12:17:57 2022] POST /identity/v3/users => generated 303 bytes in 359 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 12:17:57.798919 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68811167-2d2b-4dcd-8317-88b3eede486f None None] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:17:57.808760 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7650654d-21c8-4e80-93bd-1c71b30d7b07 admin admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:17:57.823019 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7d59fa-0b30-4e3a-b51b-7b27a3dab11b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:17:57.827906 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7d59fa-0b30-4e3a-b51b-7b27a3dab11b admin admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:17:57.828294 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7d59fa-0b30-4e3a-b51b-7b27a3dab11b admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:17:57.828584 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7d59fa-0b30-4e3a-b51b-7b27a3dab11b admin admin] PATH_INFO: `/v3/domains` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:17:57.829382 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7d59fa-0b30-4e3a-b51b-7b27a3dab11b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:17:57.829752 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7d59fa-0b30-4e3a-b51b-7b27a3dab11b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:17:57.836260 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7650654d-21c8-4e80-93bd-1c71b30d7b07 admin admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:17:57.841114 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7d59fa-0b30-4e3a-b51b-7b27a3dab11b admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:17:57.862212 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ec4e76-0221-474c-a367-6798d9fb1c39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:17:57.862923 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ec4e76-0221-474c-a367-6798d9fb1c39 admin admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:17:57.863213 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ec4e76-0221-474c-a367-6798d9fb1c39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:17:57.863496 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ec4e76-0221-474c-a367-6798d9fb1c39 admin admin] PATH_INFO: `/v3/roles` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:17:57.871077 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ec4e76-0221-474c-a367-6798d9fb1c39 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:17:57.871077 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ec4e76-0221-474c-a367-6798d9fb1c39 admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:17:57.900136 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 830/1673] 10.222.0.23 () {58 vars in 1152 bytes} [Thu May 19 12:17:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:17:57.901172 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7d59fa-0b30-4e3a-b51b-7b27a3dab11b admin admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:17:57.922918 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 844/1674] 10.222.0.23 () {58 vars in 1121 bytes} [Thu May 19 12:17:57 2022] GET /identity/v3/roles => generated 1420 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 12:17:57.935498 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ec4e76-0221-474c-a367-6798d9fb1c39 admin admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:17:57.948456 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7d59fa-0b30-4e3a-b51b-7b27a3dab11b admin admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:17:57.962170 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26cf4823-e584-4654-ad63-95c80ea94cc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:17:57.962874 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cf4823-e584-4654-ad63-95c80ea94cc5 admin admin] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:17:57.963165 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cf4823-e584-4654-ad63-95c80ea94cc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:17:57.965661 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cf4823-e584-4654-ad63-95c80ea94cc5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:17:57.966487 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26cf4823-e584-4654-ad63-95c80ea94cc5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:17:57.972394 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ec4e76-0221-474c-a367-6798d9fb1c39 admin admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:17:57.977320 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26cf4823-e584-4654-ad63-95c80ea94cc5 admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:17:57.991188 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da83c6e-abf8-41dd-a85d-d1e76b2e563d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:17:57.992980 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da83c6e-abf8-41dd-a85d-d1e76b2e563d admin admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:17:57.993279 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da83c6e-abf8-41dd-a85d-d1e76b2e563d admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:17:57.993565 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da83c6e-abf8-41dd-a85d-d1e76b2e563d admin admin] PATH_INFO: `/v3/roles` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:17:57.994322 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da83c6e-abf8-41dd-a85d-d1e76b2e563d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:17:57.997849 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da83c6e-abf8-41dd-a85d-d1e76b2e563d admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:17:58.065137 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 845/1675] 10.222.0.23 () {58 vars in 1121 bytes} [Thu May 19 12:17:57 2022] GET /identity/v3/roles => generated 1420 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 12:17:58.077983 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da83c6e-abf8-41dd-a85d-d1e76b2e563d admin admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:17:58.104963 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da83c6e-abf8-41dd-a85d-d1e76b2e563d admin admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:17:58.119547 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d17d83d-c78e-4dd4-bef7-13384a3d03ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:17:58.120750 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d17d83d-c78e-4dd4-bef7-13384a3d03ce admin admin] REQUEST_METHOD: `PUT` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:17:58.122656 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d17d83d-c78e-4dd4-bef7-13384a3d03ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:17:58.122960 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d17d83d-c78e-4dd4-bef7-13384a3d03ce admin admin] PATH_INFO: `/v3/projects/f8ff69832f884971bfaa8fcbabb6f059/users/23cf1bed54db44c29d91fb87d40c9f25/roles/259eb400cdb4439db72d14456ae9cd08` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:17:58.176620 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d17d83d-c78e-4dd4-bef7-13384a3d03ce admin admin] RBAC: Authorizing `identity:create_grant(project_id=f8ff69832f884971bfaa8fcbabb6f059, user_id=23cf1bed54db44c29d91fb87d40c9f25, role_id=259eb400cdb4439db72d14456ae9cd08)` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:17:58.181140 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d17d83d-c78e-4dd4-bef7-13384a3d03ce admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:17:58.222095 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 831/1676] 10.222.0.23 () {60 vars in 1152 bytes} [Thu May 19 12:17:57 2022] POST /identity/v3/projects => generated 385 bytes in 315 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 12:17:58.222095 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26cf4823-e584-4654-ad63-95c80ea94cc5 admin admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:17:58.306488 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 846/1677] 10.222.0.23 () {60 vars in 1482 bytes} [Thu May 19 12:17:58 2022] PUT /identity/v3/projects/f8ff69832f884971bfaa8fcbabb6f059/users/23cf1bed54db44c29d91fb87d40c9f25/roles/259eb400cdb4439db72d14456ae9cd08 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:17:58.313919 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62eaa54-13eb-45ae-9c9b-4d5d9a805d4c None None] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:17:58.314224 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62eaa54-13eb-45ae-9c9b-4d5d9a805d4c None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:17:58.314592 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62eaa54-13eb-45ae-9c9b-4d5d9a805d4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:17:58.359114 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.auth.core [None req-a62eaa54-13eb-45ae-9c9b-4d5d9a805d4c None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:17:58.364776 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-a62eaa54-13eb-45ae-9c9b-4d5d9a805d4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:17:58.417450 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26cf4823-e584-4654-ad63-95c80ea94cc5 admin admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:17:58.573827 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 847/1678] 10.222.0.23 () {58 vars in 957 bytes} [Thu May 19 12:17:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 261 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 12:17:58.592837 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62eaa54-13eb-45ae-9c9b-4d5d9a805d4c None None] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:17:58.595096 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-a62eaa54-13eb-45ae-9c9b-4d5d9a805d4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:17:58.660149 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:17:58.715295 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851117fd-c251-400c-ae7f-15437ffff059 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:17:58.716195 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851117fd-c251-400c-ae7f-15437ffff059 admin admin] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:17:58.716600 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851117fd-c251-400c-ae7f-15437ffff059 admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:17:58.717220 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851117fd-c251-400c-ae7f-15437ffff059 admin admin] PATH_INFO: `/v3/users` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:17:58.717894 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851117fd-c251-400c-ae7f-15437ffff059 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:17:58.720111 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851117fd-c251-400c-ae7f-15437ffff059 admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:17:58.791155 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62eaa54-13eb-45ae-9c9b-4d5d9a805d4c None None] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:17:58.796025 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ce7c39-cb77-4b75-8660-9e561a362c69 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a00b3fa7497456895e5e0bf105c4fc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd70d25bf69314820b8aaecb65576978a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:17:58.797171 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ce7c39-cb77-4b75-8660-9e561a362c69 service neutron] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:17:58.797607 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ce7c39-cb77-4b75-8660-9e561a362c69 service neutron] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:17:58.798146 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ce7c39-cb77-4b75-8660-9e561a362c69 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:17:58.823971 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ce7c39-cb77-4b75-8660-9e561a362c69 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:17:58.826120 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ce7c39-cb77-4b75-8660-9e561a362c69 service neutron] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:17:58.839743 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 848/1679] 10.222.0.23 () {60 vars in 1365 bytes} [Thu May 19 12:17:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 247 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:17:58.915939 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ce7c39-cb77-4b75-8660-9e561a362c69 service neutron] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:17:58.917395 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 832/1680] 10.222.0.23 () {60 vars in 1143 bytes} [Thu May 19 12:17:58 2022] POST /identity/v3/users => generated 293 bytes in 698 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 12:17:58.940424 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ce7c39-cb77-4b75-8660-9e561a362c69 service neutron] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:17:58.952622 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d75f14c-1df9-4798-afc2-efa1ccf87e1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:17:58.954073 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d75f14c-1df9-4798-afc2-efa1ccf87e1d admin admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:17:58.954443 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d75f14c-1df9-4798-afc2-efa1ccf87e1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:17:58.954816 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d75f14c-1df9-4798-afc2-efa1ccf87e1d admin admin] PATH_INFO: `/v3/roles` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:17:58.955660 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d75f14c-1df9-4798-afc2-efa1ccf87e1d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:17:58.957631 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d75f14c-1df9-4798-afc2-efa1ccf87e1d admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:17:58.983635 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 849/1681] 10.222.0.23 () {58 vars in 1121 bytes} [Thu May 19 12:17:58 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 19 12:17:58.983871 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851117fd-c251-400c-ae7f-15437ffff059 admin admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:17:59.009110 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851117fd-c251-400c-ae7f-15437ffff059 admin admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:17:59.024434 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f08199b-5da9-4bf0-875c-36300163280a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:17:59.033043 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f08199b-5da9-4bf0-875c-36300163280a admin admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:17:59.033587 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f08199b-5da9-4bf0-875c-36300163280a admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:17:59.033975 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f08199b-5da9-4bf0-875c-36300163280a admin admin] PATH_INFO: `/v3/roles` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:17:59.038950 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f08199b-5da9-4bf0-875c-36300163280a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:17:59.051858 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f08199b-5da9-4bf0-875c-36300163280a admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:17:59.060005 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d75f14c-1df9-4798-afc2-efa1ccf87e1d admin admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:17:59.062030 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-2d75f14c-1df9-4798-afc2-efa1ccf87e1d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:17:59.084181 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 833/1682] 10.222.0.23 () {58 vars in 1121 bytes} [Thu May 19 12:17:58 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 12:17:59.088540 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d75f14c-1df9-4798-afc2-efa1ccf87e1d admin admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:17:59.089281 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f08199b-5da9-4bf0-875c-36300163280a admin admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:17:59.096362 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0ed456-1e72-470b-8ca4-d05ae8a38acd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:17:59.097374 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0ed456-1e72-470b-8ca4-d05ae8a38acd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:17:59.097612 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0ed456-1e72-470b-8ca4-d05ae8a38acd admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:17:59.097871 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0ed456-1e72-470b-8ca4-d05ae8a38acd admin admin] PATH_INFO: `/v3/projects/25ad8063b2a84a529dfa0e5a5f1a8597` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:17:59.099359 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0ed456-1e72-470b-8ca4-d05ae8a38acd admin admin] RBAC: Authorizing `identity:delete_project(project_id=25ad8063b2a84a529dfa0e5a5f1a8597)` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:17:59.105582 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0ed456-1e72-470b-8ca4-d05ae8a38acd admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:17:59.114804 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f08199b-5da9-4bf0-875c-36300163280a admin admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:17:59.135433 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366d551d-c51c-43fe-b1e1-a20f5fda450f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:17:59.142902 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366d551d-c51c-43fe-b1e1-a20f5fda450f admin admin] REQUEST_METHOD: `PUT` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:17:59.154514 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366d551d-c51c-43fe-b1e1-a20f5fda450f admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:17:59.155011 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366d551d-c51c-43fe-b1e1-a20f5fda450f admin admin] PATH_INFO: `/v3/projects/65edf999e3b24c29815d321084d62bca/users/f5c2c303e27d4b30bec16808677ac66a/roles/259eb400cdb4439db72d14456ae9cd08` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:17:59.180649 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:17:59.192599 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366d551d-c51c-43fe-b1e1-a20f5fda450f admin admin] RBAC: Authorizing `identity:create_grant(project_id=65edf999e3b24c29815d321084d62bca, user_id=f5c2c303e27d4b30bec16808677ac66a, role_id=259eb400cdb4439db72d14456ae9cd08)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:17:59.197577 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366d551d-c51c-43fe-b1e1-a20f5fda450f admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:17:59.327335 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 834/1683] 10.222.0.23 () {60 vars in 1482 bytes} [Thu May 19 12:17:59 2022] PUT /identity/v3/projects/65edf999e3b24c29815d321084d62bca/users/f5c2c303e27d4b30bec16808677ac66a/roles/259eb400cdb4439db72d14456ae9cd08 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 12:17:59.352532 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1c59ab-51f1-4ba4-972f-228b4e4aab67 None None] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:17:59.353721 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1c59ab-51f1-4ba4-972f-228b4e4aab67 None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:17:59.356122 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1c59ab-51f1-4ba4-972f-228b4e4aab67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:17:59.385497 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-4f0ed456-1e72-470b-8ca4-d05ae8a38acd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '25ad8063b2a84a529dfa0e5a5f1a8597', 'request_id': 'req-4f0ed456-1e72-470b-8ca4-d05ae8a38acd'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:17:59.402646 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-4f0ed456-1e72-470b-8ca4-d05ae8a38acd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '25ad8063b2a84a529dfa0e5a5f1a8597', 'request_id': 'req-4f0ed456-1e72-470b-8ca4-d05ae8a38acd'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:17:59.412037 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-4f0ed456-1e72-470b-8ca4-d05ae8a38acd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '25ad8063b2a84a529dfa0e5a5f1a8597', 'request_id': 'req-4f0ed456-1e72-470b-8ca4-d05ae8a38acd'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:17:59.552423 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-4f0ed456-1e72-470b-8ca4-d05ae8a38acd admin admin] The token cache is being invalidate because project 25ad8063b2a84a529dfa0e5a5f1a8597 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80540) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 12:17:59.553082 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-4f0ed456-1e72-470b-8ca4-d05ae8a38acd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:17:59.554752 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-4f0ed456-1e72-470b-8ca4-d05ae8a38acd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:17:59.566805 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 850/1684] 10.222.0.23 () {58 vars in 1232 bytes} [Thu May 19 12:17:59 2022] DELETE /identity/v3/projects/25ad8063b2a84a529dfa0e5a5f1a8597 => generated 0 bytes in 507 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:17:59.578357 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfde2f2-81e9-4f74-ae57-91993ebeda6c None None] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:17:59.578701 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfde2f2-81e9-4f74-ae57-91993ebeda6c None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:17:59.579147 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfde2f2-81e9-4f74-ae57-91993ebeda6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:17:59.632779 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.auth.core [None req-1a1c59ab-51f1-4ba4-972f-228b4e4aab67 None None] MFA Rules not processed for user `8e445569dc5641ed962b0ac977850a6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:17:59.634846 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-1a1c59ab-51f1-4ba4-972f-228b4e4aab67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:17:59.642188 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.auth.core [None req-4bfde2f2-81e9-4f74-ae57-91993ebeda6c None None] MFA Rules not processed for user `f5c2c303e27d4b30bec16808677ac66a`. Rule list: `[]` (Enabled: `True`). {{(pid=80540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:17:59.643500 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-4bfde2f2-81e9-4f74-ae57-91993ebeda6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:17:59.786408 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 851/1685] 10.222.0.23 () {58 vars in 957 bytes} [Thu May 19 12:17:59 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:17:59.787840 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfde2f2-81e9-4f74-ae57-91993ebeda6c None None] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:17:59.791368 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-4bfde2f2-81e9-4f74-ae57-91993ebeda6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:17:59.831948 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 835/1686] 10.222.0.23 () {58 vars in 957 bytes} [Thu May 19 12:17:59 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 505 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:17:59.834129 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1c59ab-51f1-4ba4-972f-228b4e4aab67 None None] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:17:59.844284 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-1a1c59ab-51f1-4ba4-972f-228b4e4aab67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:17:59.924194 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfde2f2-81e9-4f74-ae57-91993ebeda6c None None] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:17:59.943330 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b596ec-044b-4170-81e5-30c0d33838a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:17:59.945578 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b596ec-044b-4170-81e5-30c0d33838a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:17:59.945578 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b596ec-044b-4170-81e5-30c0d33838a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:17:59.946646 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b596ec-044b-4170-81e5-30c0d33838a4 admin admin] PATH_INFO: `/v3/users/8862cd4c12ac45fe9f158427515947c3` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:17:59.953511 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b596ec-044b-4170-81e5-30c0d33838a4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8862cd4c12ac45fe9f158427515947c3)` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:17:59.956134 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b596ec-044b-4170-81e5-30c0d33838a4 admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:18:00.042565 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1c59ab-51f1-4ba4-972f-228b4e4aab67 None None] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:18:00.054649 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfc5730-20f8-4e95-b1ec-0a9ca0946fee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:18:00.056718 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfc5730-20f8-4e95-b1ec-0a9ca0946fee admin admin] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:18:00.057291 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfc5730-20f8-4e95-b1ec-0a9ca0946fee admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:18:00.058404 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfc5730-20f8-4e95-b1ec-0a9ca0946fee admin admin] PATH_INFO: `/v3/projects` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:18:00.059962 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dfc5730-20f8-4e95-b1ec-0a9ca0946fee admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:18:00.065522 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dfc5730-20f8-4e95-b1ec-0a9ca0946fee admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:18:00.215376 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-82b596ec-044b-4170-81e5-30c0d33838a4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8862cd4c12ac45fe9f158427515947c3'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:18:00.233995 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 836/1687] 10.222.0.23 () {60 vars in 1152 bytes} [Thu May 19 12:17:59 2022] POST /identity/v3/projects => generated 387 bytes in 401 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 12:18:00.236927 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529c0ac1-7ba4-464a-a543-c40963febe18 None None] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:18:00.237525 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529c0ac1-7ba4-464a-a543-c40963febe18 None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:18:00.238023 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529c0ac1-7ba4-464a-a543-c40963febe18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:18:00.239465 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-82b596ec-044b-4170-81e5-30c0d33838a4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8862cd4c12ac45fe9f158427515947c3'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:18:00.240551 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-82b596ec-044b-4170-81e5-30c0d33838a4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8862cd4c12ac45fe9f158427515947c3'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:18:00.259083 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:18:00.281749 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.auth.core [None req-529c0ac1-7ba4-464a-a543-c40963febe18 None None] MFA Rules not processed for user `8e445569dc5641ed962b0ac977850a6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:18:00.285597 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-529c0ac1-7ba4-464a-a543-c40963febe18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:18:00.293827 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:18:00.365390 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-82b596ec-044b-4170-81e5-30c0d33838a4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8862cd4c12ac45fe9f158427515947c3'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:18:00.384274 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-82b596ec-044b-4170-81e5-30c0d33838a4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8862cd4c12ac45fe9f158427515947c3'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:18:00.404251 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 852/1688] 10.222.0.23 () {58 vars in 1223 bytes} [Thu May 19 12:17:59 2022] DELETE /identity/v3/users/8862cd4c12ac45fe9f158427515947c3 => generated 0 bytes in 617 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:18:00.412043 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b596ec-044b-4170-81e5-30c0d33838a4 admin admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:18:00.415277 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-82b596ec-044b-4170-81e5-30c0d33838a4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:18:00.419303 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 837/1689] 10.222.0.23 () {58 vars in 957 bytes} [Thu May 19 12:18:00 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:18:00.428177 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529c0ac1-7ba4-464a-a543-c40963febe18 None None] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:18:00.430643 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-529c0ac1-7ba4-464a-a543-c40963febe18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:18:00.671867 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b596ec-044b-4170-81e5-30c0d33838a4 admin admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:18:00.683244 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529c0ac1-7ba4-464a-a543-c40963febe18 None None] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:18:00.686806 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d04231-f741-45ce-914f-3285bf637e5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:18:00.690472 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d04231-f741-45ce-914f-3285bf637e5e admin admin] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:18:00.690910 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d04231-f741-45ce-914f-3285bf637e5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:18:00.691443 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d04231-f741-45ce-914f-3285bf637e5e admin admin] PATH_INFO: `/v3/users` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:18:00.694300 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d04231-f741-45ce-914f-3285bf637e5e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:18:00.698578 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d04231-f741-45ce-914f-3285bf637e5e admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:18:00.817326 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa39862-e712-42ce-838e-85572d1a1c87 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34a98d75d84b4c67981377e0f26eca62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd70d25bf69314820b8aaecb65576978a', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:18:00.818672 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa39862-e712-42ce-838e-85572d1a1c87 service nova] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:18:00.819159 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa39862-e712-42ce-838e-85572d1a1c87 service nova] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:18:00.820254 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa39862-e712-42ce-838e-85572d1a1c87 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:18:00.826045 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-1aa39862-e712-42ce-838e-85572d1a1c87 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:18:00.845621 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 853/1690] 10.222.0.23 () {60 vars in 1143 bytes} [Thu May 19 12:18:00 2022] POST /identity/v3/users => generated 300 bytes in 435 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 12:18:00.871453 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d04231-f741-45ce-914f-3285bf637e5e admin admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:18:00.902613 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d04231-f741-45ce-914f-3285bf637e5e admin admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:18:00.919195 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81445d45-7e31-4c89-9396-7fe49f529544 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:18:00.919936 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81445d45-7e31-4c89-9396-7fe49f529544 admin admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:18:00.920228 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81445d45-7e31-4c89-9396-7fe49f529544 admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:18:00.920514 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81445d45-7e31-4c89-9396-7fe49f529544 admin admin] PATH_INFO: `/v3/roles` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:18:00.921318 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81445d45-7e31-4c89-9396-7fe49f529544 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:18:00.923113 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81445d45-7e31-4c89-9396-7fe49f529544 admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:18:00.960716 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 854/1691] 10.222.0.23 () {58 vars in 1121 bytes} [Thu May 19 12:18:00 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 12:18:00.964979 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81445d45-7e31-4c89-9396-7fe49f529544 admin admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:18:00.991354 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81445d45-7e31-4c89-9396-7fe49f529544 admin admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:18:00.998991 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79028ee-7b40-4e85-8281-fbab6469e689 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:18:00.999933 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79028ee-7b40-4e85-8281-fbab6469e689 admin admin] REQUEST_METHOD: `PUT` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:18:01.000221 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79028ee-7b40-4e85-8281-fbab6469e689 admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:18:01.000503 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79028ee-7b40-4e85-8281-fbab6469e689 admin admin] PATH_INFO: `/v3/projects/d5a205482a6c4dd8b18cd749bc2e45ff/users/798f925e652944e089359754e0181e7a/roles/9db5671c65d3414ea2682afa2ccf1403` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:18:01.033987 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa39862-e712-42ce-838e-85572d1a1c87 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:18:01.036029 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79028ee-7b40-4e85-8281-fbab6469e689 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d5a205482a6c4dd8b18cd749bc2e45ff, user_id=798f925e652944e089359754e0181e7a, role_id=9db5671c65d3414ea2682afa2ccf1403)` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:18:01.037816 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa39862-e712-42ce-838e-85572d1a1c87 service nova] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:18:01.039152 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79028ee-7b40-4e85-8281-fbab6469e689 admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:18:01.054806 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 838/1692] 10.222.0.23 () {60 vars in 1365 bytes} [Thu May 19 12:18:00 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 627 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 12:18:01.176800 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 855/1693] 10.222.0.23 () {60 vars in 1482 bytes} [Thu May 19 12:18:00 2022] PUT /identity/v3/projects/d5a205482a6c4dd8b18cd749bc2e45ff/users/798f925e652944e089359754e0181e7a/roles/9db5671c65d3414ea2682afa2ccf1403 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 12:18:01.181818 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa39862-e712-42ce-838e-85572d1a1c87 service nova] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:18:01.212434 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa39862-e712-42ce-838e-85572d1a1c87 service nova] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:18:01.228909 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:18:01.390390 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae02019-ded4-4e77-807e-b1589cb27982 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:18:01.390938 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae02019-ded4-4e77-807e-b1589cb27982 admin admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:18:01.391233 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae02019-ded4-4e77-807e-b1589cb27982 admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:18:01.391514 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae02019-ded4-4e77-807e-b1589cb27982 admin admin] PATH_INFO: `/v3/roles` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:18:01.394826 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae02019-ded4-4e77-807e-b1589cb27982 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:18:01.399338 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae02019-ded4-4e77-807e-b1589cb27982 admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:18:01.433255 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79028ee-7b40-4e85-8281-fbab6469e689 admin admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:18:01.447001 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 839/1694] 10.222.0.23 () {58 vars in 1121 bytes} [Thu May 19 12:18:01 2022] GET /identity/v3/roles => generated 1420 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 12:18:01.455389 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79028ee-7b40-4e85-8281-fbab6469e689 admin admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:18:01.462668 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf44c3c-ccd0-4696-b771-6877a051efcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:18:01.463985 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf44c3c-ccd0-4696-b771-6877a051efcd admin admin] REQUEST_METHOD: `PUT` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:18:01.464977 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf44c3c-ccd0-4696-b771-6877a051efcd admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:18:01.464977 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf44c3c-ccd0-4696-b771-6877a051efcd admin admin] PATH_INFO: `/v3/projects/d5a205482a6c4dd8b18cd749bc2e45ff/users/798f925e652944e089359754e0181e7a/roles/9db5671c65d3414ea2682afa2ccf1403` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:18:01.467319 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf44c3c-ccd0-4696-b771-6877a051efcd admin admin] RBAC: Authorizing `identity:create_grant(project_id=d5a205482a6c4dd8b18cd749bc2e45ff, user_id=798f925e652944e089359754e0181e7a, role_id=9db5671c65d3414ea2682afa2ccf1403)` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:18:01.470725 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf44c3c-ccd0-4696-b771-6877a051efcd admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:18:01.486116 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 856/1695] 10.222.0.23 () {60 vars in 1482 bytes} [Thu May 19 12:18:01 2022] PUT /identity/v3/projects/d5a205482a6c4dd8b18cd749bc2e45ff/users/798f925e652944e089359754e0181e7a/roles/9db5671c65d3414ea2682afa2ccf1403 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:18:01.502110 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f927e09-0f76-4bbd-ae62-e364ceb16634 None None] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:18:01.502658 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f927e09-0f76-4bbd-ae62-e364ceb16634 None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:18:01.503396 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f927e09-0f76-4bbd-ae62-e364ceb16634 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:18:01.578754 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.auth.core [None req-1f927e09-0f76-4bbd-ae62-e364ceb16634 None None] MFA Rules not processed for user `798f925e652944e089359754e0181e7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:18:01.579510 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-1f927e09-0f76-4bbd-ae62-e364ceb16634 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:18:01.699346 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 840/1696] 10.222.0.23 () {58 vars in 957 bytes} [Thu May 19 12:18:01 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:18:01.719599 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf44c3c-ccd0-4696-b771-6877a051efcd admin admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:18:01.721592 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-4bf44c3c-ccd0-4696-b771-6877a051efcd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:18:01.892012 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf44c3c-ccd0-4696-b771-6877a051efcd admin admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:18:01.902137 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ff72b8-20e9-4d00-a709-401eb015d576 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8800254c0e784e42a791fb9abc274d79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd70d25bf69314820b8aaecb65576978a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:18:01.902815 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ff72b8-20e9-4d00-a709-401eb015d576 service cinder] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:18:01.903233 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ff72b8-20e9-4d00-a709-401eb015d576 service cinder] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:18:01.903539 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ff72b8-20e9-4d00-a709-401eb015d576 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:18:01.924533 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ff72b8-20e9-4d00-a709-401eb015d576 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:18:01.926042 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ff72b8-20e9-4d00-a709-401eb015d576 service cinder] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:18:01.949626 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 857/1697] 10.222.0.23 () {60 vars in 1365 bytes} [Thu May 19 12:18:01 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 230 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:18:02.889220 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f927e09-0f76-4bbd-ae62-e364ceb16634 None None] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:18:02.891635 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-1f927e09-0f76-4bbd-ae62-e364ceb16634 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:18:03.044472 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f927e09-0f76-4bbd-ae62-e364ceb16634 None None] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:18:03.050870 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066d01a5-01d8-4aeb-991b-be48cd553aa0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:18:03.052108 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066d01a5-01d8-4aeb-991b-be48cd553aa0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:18:03.052644 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066d01a5-01d8-4aeb-991b-be48cd553aa0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:18:03.053051 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066d01a5-01d8-4aeb-991b-be48cd553aa0 admin admin] PATH_INFO: `/v3/projects/96b07d3cf30e4789947e5a79b440d720` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:18:03.055507 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066d01a5-01d8-4aeb-991b-be48cd553aa0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=96b07d3cf30e4789947e5a79b440d720)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:18:03.075242 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066d01a5-01d8-4aeb-991b-be48cd553aa0 admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:18:03.112939 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:18:03.367462 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-066d01a5-01d8-4aeb-991b-be48cd553aa0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '96b07d3cf30e4789947e5a79b440d720', 'request_id': 'req-066d01a5-01d8-4aeb-991b-be48cd553aa0'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:18:03.376084 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96ef1e2-d973-468b-acb6-2d7b4d59fd86 None None] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:18:03.376793 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96ef1e2-d973-468b-acb6-2d7b4d59fd86 None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:18:03.376793 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96ef1e2-d973-468b-acb6-2d7b4d59fd86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:18:03.380663 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-066d01a5-01d8-4aeb-991b-be48cd553aa0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '96b07d3cf30e4789947e5a79b440d720', 'request_id': 'req-066d01a5-01d8-4aeb-991b-be48cd553aa0'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:18:03.388359 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=90,keystone:DELETE=4,keystone:INSERT=6 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:18:03.424640 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.auth.core [None req-b96ef1e2-d973-468b-acb6-2d7b4d59fd86 None None] MFA Rules not processed for user `0c3181fb4f584a549a786adfefdf3dfb`. Rule list: `[]` (Enabled: `True`). {{(pid=80540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:18:03.432133 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-b96ef1e2-d973-468b-acb6-2d7b4d59fd86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:18:03.515179 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-066d01a5-01d8-4aeb-991b-be48cd553aa0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '96b07d3cf30e4789947e5a79b440d720', 'request_id': 'req-066d01a5-01d8-4aeb-991b-be48cd553aa0'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:18:03.546562 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-066d01a5-01d8-4aeb-991b-be48cd553aa0 admin admin] The token cache is being invalidate because project 96b07d3cf30e4789947e5a79b440d720 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80539) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 12:18:03.547052 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-066d01a5-01d8-4aeb-991b-be48cd553aa0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:18:03.549580 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-066d01a5-01d8-4aeb-991b-be48cd553aa0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:18:03.560753 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 841/1698] 10.222.0.23 () {58 vars in 1232 bytes} [Thu May 19 12:18:02 2022] DELETE /identity/v3/projects/96b07d3cf30e4789947e5a79b440d720 => generated 0 bytes in 672 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:18:03.565437 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c782ff8-d9a9-42d0-9a39-b3485069d4bd None None] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:18:03.570500 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c782ff8-d9a9-42d0-9a39-b3485069d4bd None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:18:03.571299 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c782ff8-d9a9-42d0-9a39-b3485069d4bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:18:03.686186 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 858/1699] 10.222.0.23 () {58 vars in 1009 bytes} [Thu May 19 12:18:03 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 311 msecs (HTTP/1.1 201) 6 headers in 385 bytes (3 switches on core 0) May 19 12:18:03.694425 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf71cbd-b709-492d-8fb6-2a4fb8f3a67f None None] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:18:03.696381 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf71cbd-b709-492d-8fb6-2a4fb8f3a67f None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:18:03.696998 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf71cbd-b709-492d-8fb6-2a4fb8f3a67f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:18:03.728414 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.auth.core [None req-8c782ff8-d9a9-42d0-9a39-b3485069d4bd None None] MFA Rules not processed for user `23cf1bed54db44c29d91fb87d40c9f25`. Rule list: `[]` (Enabled: `True`). {{(pid=80539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:18:03.744756 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-8c782ff8-d9a9-42d0-9a39-b3485069d4bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:18:03.825364 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.auth.core [None req-4bf71cbd-b709-492d-8fb6-2a4fb8f3a67f None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:18:03.826922 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-4bf71cbd-b709-492d-8fb6-2a4fb8f3a67f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:18:04.065020 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 842/1700] 10.222.0.23 () {58 vars in 957 bytes} [Thu May 19 12:18:03 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 503 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:18:04.068363 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 859/1701] 10.222.0.23 () {58 vars in 957 bytes} [Thu May 19 12:18:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 381 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:18:04.075147 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c782ff8-d9a9-42d0-9a39-b3485069d4bd None None] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:18:04.076936 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9753f0b-722c-45cb-aa10-c1ad84fa6afe None None] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:18:04.077342 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9753f0b-722c-45cb-aa10-c1ad84fa6afe None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:18:04.077507 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-8c782ff8-d9a9-42d0-9a39-b3485069d4bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:18:04.077737 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9753f0b-722c-45cb-aa10-c1ad84fa6afe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:18:04.120987 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c782ff8-d9a9-42d0-9a39-b3485069d4bd None None] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:18:04.126890 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.auth.core [None req-c9753f0b-722c-45cb-aa10-c1ad84fa6afe None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:18:04.128754 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-c9753f0b-722c-45cb-aa10-c1ad84fa6afe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:18:04.132911 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7e4c5d-89d3-4a14-933d-78e7cb20144f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:18:04.134356 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7e4c5d-89d3-4a14-933d-78e7cb20144f admin admin] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:18:04.134677 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7e4c5d-89d3-4a14-933d-78e7cb20144f admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:18:04.134966 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7e4c5d-89d3-4a14-933d-78e7cb20144f admin admin] PATH_INFO: `/v3/projects` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:18:04.140179 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7e4c5d-89d3-4a14-933d-78e7cb20144f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:18:04.141626 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 860/1702] 10.222.0.23 () {58 vars in 957 bytes} [Thu May 19 12:18:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 67 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:18:04.149076 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7e4c5d-89d3-4a14-933d-78e7cb20144f admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:18:04.149547 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9753f0b-722c-45cb-aa10-c1ad84fa6afe None None] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:18:04.209366 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9753f0b-722c-45cb-aa10-c1ad84fa6afe None None] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:18:04.249271 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32f3622-f9e2-432d-b19a-2d8d9f2df550 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:18:04.254112 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32f3622-f9e2-432d-b19a-2d8d9f2df550 admin admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:18:04.254412 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32f3622-f9e2-432d-b19a-2d8d9f2df550 admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:18:04.254887 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32f3622-f9e2-432d-b19a-2d8d9f2df550 admin admin] PATH_INFO: `/v3/domains` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:18:04.256509 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32f3622-f9e2-432d-b19a-2d8d9f2df550 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:18:04.257043 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32f3622-f9e2-432d-b19a-2d8d9f2df550 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:18:04.259988 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32f3622-f9e2-432d-b19a-2d8d9f2df550 admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:18:04.302991 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 861/1703] 10.222.0.23 () {58 vars in 1152 bytes} [Thu May 19 12:18:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:18:04.345944 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32f3622-f9e2-432d-b19a-2d8d9f2df550 admin admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:18:04.346614 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 843/1704] 10.222.0.23 () {60 vars in 1152 bytes} [Thu May 19 12:18:04 2022] POST /identity/v3/projects => generated 403 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 12:18:04.346614 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7e4c5d-89d3-4a14-933d-78e7cb20144f admin admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:18:04.346775 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-e32f3622-f9e2-432d-b19a-2d8d9f2df550 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:18:04.346853 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-3b7e4c5d-89d3-4a14-933d-78e7cb20144f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:18:04.401833 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32f3622-f9e2-432d-b19a-2d8d9f2df550 admin admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:18:04.597470 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a44489-9de7-4d8d-ae71-91cd0e9c9a53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:18:04.603582 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a44489-9de7-4d8d-ae71-91cd0e9c9a53 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:18:04.604282 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a44489-9de7-4d8d-ae71-91cd0e9c9a53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:18:04.604717 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a44489-9de7-4d8d-ae71-91cd0e9c9a53 admin admin] PATH_INFO: `/v3/users/f5c2c303e27d4b30bec16808677ac66a` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:18:04.606312 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a44489-9de7-4d8d-ae71-91cd0e9c9a53 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f5c2c303e27d4b30bec16808677ac66a)` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:18:04.609168 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a44489-9de7-4d8d-ae71-91cd0e9c9a53 admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:18:04.680088 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7e4c5d-89d3-4a14-933d-78e7cb20144f admin admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:18:04.713271 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0210a878-c4af-4e9c-bc77-dc8a08e9d1f7 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15cbfc73ae1243d2a8b1e70f9cd99876', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd70d25bf69314820b8aaecb65576978a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:18:04.713984 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0210a878-c4af-4e9c-bc77-dc8a08e9d1f7 service swift] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:18:04.714275 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0210a878-c4af-4e9c-bc77-dc8a08e9d1f7 service swift] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:18:04.714557 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0210a878-c4af-4e9c-bc77-dc8a08e9d1f7 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:18:04.734051 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:18:04.825361 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0210a878-c4af-4e9c-bc77-dc8a08e9d1f7 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:18:04.828582 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0210a878-c4af-4e9c-bc77-dc8a08e9d1f7 service swift] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:18:04.865715 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 844/1705] 10.222.0.23 () {60 vars in 1384 bytes} [Thu May 19 12:18:04 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 536 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 19 12:18:04.867251 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0210a878-c4af-4e9c-bc77-dc8a08e9d1f7 service swift] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:18:04.881712 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0210a878-c4af-4e9c-bc77-dc8a08e9d1f7 service swift] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:18:04.888434 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b0ee38-44f8-430b-9792-1a4299d94466 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:18:04.888939 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b0ee38-44f8-430b-9792-1a4299d94466 admin admin] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:18:04.889428 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b0ee38-44f8-430b-9792-1a4299d94466 admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:18:04.890267 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b0ee38-44f8-430b-9792-1a4299d94466 admin admin] PATH_INFO: `/v3/projects` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:18:04.892373 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b0ee38-44f8-430b-9792-1a4299d94466 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:18:04.894936 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b0ee38-44f8-430b-9792-1a4299d94466 admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:18:04.910405 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:18:04.941887 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-14a44489-9de7-4d8d-ae71-91cd0e9c9a53 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f5c2c303e27d4b30bec16808677ac66a'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:18:04.969411 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-14a44489-9de7-4d8d-ae71-91cd0e9c9a53 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f5c2c303e27d4b30bec16808677ac66a'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:18:04.969411 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-14a44489-9de7-4d8d-ae71-91cd0e9c9a53 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f5c2c303e27d4b30bec16808677ac66a'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:18:05.053454 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-14a44489-9de7-4d8d-ae71-91cd0e9c9a53 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f5c2c303e27d4b30bec16808677ac66a'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:18:05.087207 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 845/1706] 10.222.0.23 () {60 vars in 1152 bytes} [Thu May 19 12:18:04 2022] POST /identity/v3/projects => generated 367 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 12:18:05.089650 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-14a44489-9de7-4d8d-ae71-91cd0e9c9a53 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f5c2c303e27d4b30bec16808677ac66a'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:18:05.091584 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b0ee38-44f8-430b-9792-1a4299d94466 admin admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:18:05.103689 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-d9b0ee38-44f8-430b-9792-1a4299d94466 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:18:05.126840 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 862/1707] 10.222.0.23 () {58 vars in 1223 bytes} [Thu May 19 12:18:04 2022] DELETE /identity/v3/users/f5c2c303e27d4b30bec16808677ac66a => generated 0 bytes in 824 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:18:05.133538 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629d9d08-8434-4b38-a71d-f13941ea6b32 None None] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:18:05.134089 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629d9d08-8434-4b38-a71d-f13941ea6b32 None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:18:05.134258 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629d9d08-8434-4b38-a71d-f13941ea6b32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:18:05.277834 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b0ee38-44f8-430b-9792-1a4299d94466 admin admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:18:05.296593 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.auth.core [None req-629d9d08-8434-4b38-a71d-f13941ea6b32 None None] MFA Rules not processed for user `8e445569dc5641ed962b0ac977850a6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:18:05.303121 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-629d9d08-8434-4b38-a71d-f13941ea6b32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:18:05.597884 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22742ee5-faf6-4561-8e2c-158161e4c51e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:18:05.608351 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22742ee5-faf6-4561-8e2c-158161e4c51e admin admin] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:18:05.608736 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22742ee5-faf6-4561-8e2c-158161e4c51e admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:18:05.617701 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22742ee5-faf6-4561-8e2c-158161e4c51e admin admin] PATH_INFO: `/v3/users` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:18:05.639412 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 863/1708] 10.222.0.23 () {58 vars in 957 bytes} [Thu May 19 12:18:05 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 515 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:18:05.640955 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629d9d08-8434-4b38-a71d-f13941ea6b32 None None] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:18:05.646477 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22742ee5-faf6-4561-8e2c-158161e4c51e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:18:05.649141 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22742ee5-faf6-4561-8e2c-158161e4c51e admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:18:05.660075 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-629d9d08-8434-4b38-a71d-f13941ea6b32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:18:05.707217 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629d9d08-8434-4b38-a71d-f13941ea6b32 None None] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:18:05.718500 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f85497-c737-4cc3-85ec-4fd3bea8d760 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:18:05.719396 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f85497-c737-4cc3-85ec-4fd3bea8d760 admin admin] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:18:05.719831 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f85497-c737-4cc3-85ec-4fd3bea8d760 admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:18:05.720182 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f85497-c737-4cc3-85ec-4fd3bea8d760 admin admin] PATH_INFO: `/v3/users` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:18:05.721059 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f85497-c737-4cc3-85ec-4fd3bea8d760 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:18:05.723225 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f85497-c737-4cc3-85ec-4fd3bea8d760 admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:18:05.854254 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 846/1709] 10.222.0.23 () {60 vars in 1143 bytes} [Thu May 19 12:18:05 2022] POST /identity/v3/users => generated 308 bytes in 767 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 12:18:05.859830 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b089987-ccf1-4b87-9c5b-28e08c3a5e74 None None] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:18:05.862804 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b089987-ccf1-4b87-9c5b-28e08c3a5e74 None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:18:05.864400 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b089987-ccf1-4b87-9c5b-28e08c3a5e74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:18:05.901241 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 864/1710] 10.222.0.23 () {60 vars in 1143 bytes} [Thu May 19 12:18:05 2022] POST /identity/v3/users => generated 284 bytes in 260 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 12:18:05.903153 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f85497-c737-4cc3-85ec-4fd3bea8d760 admin admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:18:05.925651 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f85497-c737-4cc3-85ec-4fd3bea8d760 admin admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:18:05.935369 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c7bc08-73ca-41a1-bd01-fb5afbf363c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:18:05.940073 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c7bc08-73ca-41a1-bd01-fb5afbf363c6 admin admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:18:05.940530 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c7bc08-73ca-41a1-bd01-fb5afbf363c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:18:05.940988 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c7bc08-73ca-41a1-bd01-fb5afbf363c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:18:05.942249 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c7bc08-73ca-41a1-bd01-fb5afbf363c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:18:05.951480 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c7bc08-73ca-41a1-bd01-fb5afbf363c6 admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:18:05.981897 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.auth.core [None req-0b089987-ccf1-4b87-9c5b-28e08c3a5e74 None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:18:05.987260 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-0b089987-ccf1-4b87-9c5b-28e08c3a5e74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:18:06.024689 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 865/1711] 10.222.0.23 () {58 vars in 1121 bytes} [Thu May 19 12:18:05 2022] GET /identity/v3/roles => generated 1420 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 12:18:06.026684 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c7bc08-73ca-41a1-bd01-fb5afbf363c6 admin admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:18:06.029443 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 847/1712] 10.222.0.23 () {58 vars in 957 bytes} [Thu May 19 12:18:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:18:06.032554 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b089987-ccf1-4b87-9c5b-28e08c3a5e74 None None] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:18:06.069451 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c7bc08-73ca-41a1-bd01-fb5afbf363c6 admin admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:18:06.077121 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b089987-ccf1-4b87-9c5b-28e08c3a5e74 None None] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:18:06.095420 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023cafdd-00a4-44e1-b8a8-050d924c6002 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:18:06.096598 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023cafdd-00a4-44e1-b8a8-050d924c6002 admin admin] REQUEST_METHOD: `PUT` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:18:06.102484 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b888cc-c392-49f5-8d38-a873eb81e1ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:18:06.105334 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023cafdd-00a4-44e1-b8a8-050d924c6002 admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:18:06.105915 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023cafdd-00a4-44e1-b8a8-050d924c6002 admin admin] PATH_INFO: `/v3/projects/96fcdc6c8ade449c828f08da5aba92e6/users/8c99000c8e6f4a98b430bfd479f87287/roles/9db5671c65d3414ea2682afa2ccf1403` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:18:06.106093 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b888cc-c392-49f5-8d38-a873eb81e1ab admin admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:18:06.106697 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b888cc-c392-49f5-8d38-a873eb81e1ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:18:06.107572 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b888cc-c392-49f5-8d38-a873eb81e1ab admin admin] PATH_INFO: `/v3/roles` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:18:06.110263 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b888cc-c392-49f5-8d38-a873eb81e1ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:18:06.117885 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b888cc-c392-49f5-8d38-a873eb81e1ab admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:18:06.157458 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023cafdd-00a4-44e1-b8a8-050d924c6002 admin admin] RBAC: Authorizing `identity:create_grant(project_id=96fcdc6c8ade449c828f08da5aba92e6, user_id=8c99000c8e6f4a98b430bfd479f87287, role_id=9db5671c65d3414ea2682afa2ccf1403)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:18:06.160903 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023cafdd-00a4-44e1-b8a8-050d924c6002 admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:18:06.166753 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:18:06.221569 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 866/1713] 10.222.0.23 () {58 vars in 1121 bytes} [Thu May 19 12:18:06 2022] GET /identity/v3/roles => generated 1420 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 12:18:06.227300 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b888cc-c392-49f5-8d38-a873eb81e1ab admin admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:18:06.245550 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-75b888cc-c392-49f5-8d38-a873eb81e1ab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:18:06.388016 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 848/1714] 10.222.0.23 () {60 vars in 1482 bytes} [Thu May 19 12:18:06 2022] PUT /identity/v3/projects/96fcdc6c8ade449c828f08da5aba92e6/users/8c99000c8e6f4a98b430bfd479f87287/roles/9db5671c65d3414ea2682afa2ccf1403 => generated 0 bytes in 358 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:18:06.389249 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023cafdd-00a4-44e1-b8a8-050d924c6002 admin admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:18:06.669492 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b888cc-c392-49f5-8d38-a873eb81e1ab admin admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:18:06.669492 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:18:06.670670 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023cafdd-00a4-44e1-b8a8-050d924c6002 admin admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:18:06.670670 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d6bce1-e391-4d69-96a5-54f915a59c62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:18:06.670670 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d6bce1-e391-4d69-96a5-54f915a59c62 admin admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:18:06.670670 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d6bce1-e391-4d69-96a5-54f915a59c62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:18:06.671033 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d6bce1-e391-4d69-96a5-54f915a59c62 admin admin] PATH_INFO: `/v3/roles` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:18:06.671033 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d6bce1-e391-4d69-96a5-54f915a59c62 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:18:06.671164 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7013a4c-ee7d-4f66-8af8-e87133214366 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a00b3fa7497456895e5e0bf105c4fc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd70d25bf69314820b8aaecb65576978a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:18:06.671247 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d6bce1-e391-4d69-96a5-54f915a59c62 admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:18:06.671324 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7013a4c-ee7d-4f66-8af8-e87133214366 service neutron] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:18:06.671324 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7013a4c-ee7d-4f66-8af8-e87133214366 service neutron] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:18:06.671324 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7013a4c-ee7d-4f66-8af8-e87133214366 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:18:06.677946 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 849/1715] 10.222.0.23 () {58 vars in 1121 bytes} [Thu May 19 12:18:06 2022] GET /identity/v3/roles => generated 1420 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 12:18:06.678561 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d6bce1-e391-4d69-96a5-54f915a59c62 admin admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:18:06.690108 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7013a4c-ee7d-4f66-8af8-e87133214366 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:18:06.709250 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7013a4c-ee7d-4f66-8af8-e87133214366 service neutron] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:18:06.715382 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d6bce1-e391-4d69-96a5-54f915a59c62 admin admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:18:06.738629 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb37b320-7e9c-458f-86ed-276e33822328 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:18:06.739317 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb37b320-7e9c-458f-86ed-276e33822328 admin admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:18:06.739619 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb37b320-7e9c-458f-86ed-276e33822328 admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:18:06.739982 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb37b320-7e9c-458f-86ed-276e33822328 admin admin] PATH_INFO: `/v3/roles` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:18:06.740847 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb37b320-7e9c-458f-86ed-276e33822328 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:18:06.751823 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb37b320-7e9c-458f-86ed-276e33822328 admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:18:06.770688 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 867/1716] 10.222.0.23 () {60 vars in 1365 bytes} [Thu May 19 12:18:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 548 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:18:06.772071 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7013a4c-ee7d-4f66-8af8-e87133214366 service neutron] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:18:06.796977 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7013a4c-ee7d-4f66-8af8-e87133214366 service neutron] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:18:06.814597 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44bb3981-6fff-482d-8962-58826b3eea92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:18:06.816371 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bb3981-6fff-482d-8962-58826b3eea92 admin admin] REQUEST_METHOD: `PUT` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:18:06.820081 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 850/1717] 10.222.0.23 () {58 vars in 1121 bytes} [Thu May 19 12:18:06 2022] GET /identity/v3/roles => generated 1420 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 12:18:06.821243 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb37b320-7e9c-458f-86ed-276e33822328 admin admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:18:06.822544 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bb3981-6fff-482d-8962-58826b3eea92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:18:06.823822 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bb3981-6fff-482d-8962-58826b3eea92 admin admin] PATH_INFO: `/v3/projects/51937f43dc4c4f6b8d174fa3dbe47baa/users/4334c63b156e44908453a320dbe0524b/roles/259eb400cdb4439db72d14456ae9cd08` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:18:06.854450 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44bb3981-6fff-482d-8962-58826b3eea92 admin admin] RBAC: Authorizing `identity:create_grant(project_id=51937f43dc4c4f6b8d174fa3dbe47baa, user_id=4334c63b156e44908453a320dbe0524b, role_id=259eb400cdb4439db72d14456ae9cd08)` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:18:06.856590 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb37b320-7e9c-458f-86ed-276e33822328 admin admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:18:06.860809 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44bb3981-6fff-482d-8962-58826b3eea92 admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:18:06.871884 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaae8435-9808-4604-849f-74468c930d69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:18:06.873790 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaae8435-9808-4604-849f-74468c930d69 admin admin] REQUEST_METHOD: `PUT` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:18:06.874720 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaae8435-9808-4604-849f-74468c930d69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:18:06.875005 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaae8435-9808-4604-849f-74468c930d69 admin admin] PATH_INFO: `/v3/projects/96fcdc6c8ade449c828f08da5aba92e6/users/8c99000c8e6f4a98b430bfd479f87287/roles/9db5671c65d3414ea2682afa2ccf1403` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:18:06.878115 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaae8435-9808-4604-849f-74468c930d69 admin admin] RBAC: Authorizing `identity:create_grant(project_id=96fcdc6c8ade449c828f08da5aba92e6, user_id=8c99000c8e6f4a98b430bfd479f87287, role_id=9db5671c65d3414ea2682afa2ccf1403)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:18:06.881247 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaae8435-9808-4604-849f-74468c930d69 admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:18:06.904797 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 851/1718] 10.222.0.23 () {60 vars in 1482 bytes} [Thu May 19 12:18:06 2022] PUT /identity/v3/projects/96fcdc6c8ade449c828f08da5aba92e6/users/8c99000c8e6f4a98b430bfd479f87287/roles/9db5671c65d3414ea2682afa2ccf1403 => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:18:07.083018 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 868/1719] 10.222.0.23 () {60 vars in 1482 bytes} [Thu May 19 12:18:06 2022] PUT /identity/v3/projects/51937f43dc4c4f6b8d174fa3dbe47baa/users/4334c63b156e44908453a320dbe0524b/roles/259eb400cdb4439db72d14456ae9cd08 => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:18:07.084955 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2909799e-53ca-4a7b-96d3-1e1fc0c6eb2a None None] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:18:07.085428 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2909799e-53ca-4a7b-96d3-1e1fc0c6eb2a None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:18:07.085945 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2909799e-53ca-4a7b-96d3-1e1fc0c6eb2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:18:07.172457 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.auth.core [None req-2909799e-53ca-4a7b-96d3-1e1fc0c6eb2a None None] MFA Rules not processed for user `4334c63b156e44908453a320dbe0524b`. Rule list: `[]` (Enabled: `True`). {{(pid=80539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:18:07.174437 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-2909799e-53ca-4a7b-96d3-1e1fc0c6eb2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:18:07.625141 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 852/1720] 10.222.0.23 () {58 vars in 957 bytes} [Thu May 19 12:18:07 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 541 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:18:07.628422 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44bb3981-6fff-482d-8962-58826b3eea92 admin admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:18:07.905432 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44bb3981-6fff-482d-8962-58826b3eea92 admin admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:18:07.923435 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6214dc1d-ec2e-4bdf-bf5b-dc6f06cfb962 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:18:07.924169 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6214dc1d-ec2e-4bdf-bf5b-dc6f06cfb962 admin admin] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:18:07.932659 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6214dc1d-ec2e-4bdf-bf5b-dc6f06cfb962 admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:18:07.932659 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6214dc1d-ec2e-4bdf-bf5b-dc6f06cfb962 admin admin] PATH_INFO: `/v3/projects` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:18:07.932659 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6214dc1d-ec2e-4bdf-bf5b-dc6f06cfb962 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:18:07.937745 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6214dc1d-ec2e-4bdf-bf5b-dc6f06cfb962 admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:18:08.077806 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 869/1721] 10.222.0.23 () {60 vars in 1152 bytes} [Thu May 19 12:18:07 2022] POST /identity/v3/projects => generated 367 bytes in 451 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 12:18:08.082948 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2909799e-53ca-4a7b-96d3-1e1fc0c6eb2a None None] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:18:08.129738 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2909799e-53ca-4a7b-96d3-1e1fc0c6eb2a None None] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:18:08.320734 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:18:08.346517 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4672afe7-0e69-4c68-99db-c55dd3e17e83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:18:08.349713 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4672afe7-0e69-4c68-99db-c55dd3e17e83 admin admin] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:18:08.350776 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4672afe7-0e69-4c68-99db-c55dd3e17e83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:18:08.354838 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4672afe7-0e69-4c68-99db-c55dd3e17e83 admin admin] PATH_INFO: `/v3/users` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:18:08.359740 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4672afe7-0e69-4c68-99db-c55dd3e17e83 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:18:08.371263 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4672afe7-0e69-4c68-99db-c55dd3e17e83 admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:18:08.485014 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394b74ac-5f64-4636-808e-219a0f5112ae None None] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:18:08.485014 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394b74ac-5f64-4636-808e-219a0f5112ae None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:18:08.485313 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394b74ac-5f64-4636-808e-219a0f5112ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:18:08.563005 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.auth.core [None req-394b74ac-5f64-4636-808e-219a0f5112ae None None] MFA Rules not processed for user `9d762cce00ae4e468c358b0437f35deb`. Rule list: `[]` (Enabled: `True`). {{(pid=80540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:18:08.564362 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-394b74ac-5f64-4636-808e-219a0f5112ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:18:08.595022 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 853/1722] 10.222.0.23 () {60 vars in 1143 bytes} [Thu May 19 12:18:08 2022] POST /identity/v3/users => generated 290 bytes in 513 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 12:18:08.615998 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4672afe7-0e69-4c68-99db-c55dd3e17e83 admin admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:18:08.694250 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4672afe7-0e69-4c68-99db-c55dd3e17e83 admin admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:18:08.706916 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 870/1723] 10.222.0.23 () {58 vars in 957 bytes} [Thu May 19 12:18:08 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:18:08.725198 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1989f99f-397c-4636-acab-4e546afd7240 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:18:08.754525 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1989f99f-397c-4636-acab-4e546afd7240 admin admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:18:08.754525 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1989f99f-397c-4636-acab-4e546afd7240 admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:18:08.754525 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1989f99f-397c-4636-acab-4e546afd7240 admin admin] PATH_INFO: `/v3/roles` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:18:08.762088 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1989f99f-397c-4636-acab-4e546afd7240 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:18:08.787953 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1989f99f-397c-4636-acab-4e546afd7240 admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:18:08.833987 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 854/1724] 10.222.0.23 () {58 vars in 1121 bytes} [Thu May 19 12:18:08 2022] GET /identity/v3/roles => generated 1420 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 12:18:08.839264 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394b74ac-5f64-4636-808e-219a0f5112ae None None] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:18:08.863147 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394b74ac-5f64-4636-808e-219a0f5112ae None None] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:18:08.885518 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef624b4-fa4a-4ee8-9070-7674ac7ab8d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:18:08.889077 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef624b4-fa4a-4ee8-9070-7674ac7ab8d9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:18:08.889748 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef624b4-fa4a-4ee8-9070-7674ac7ab8d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:18:08.894412 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef624b4-fa4a-4ee8-9070-7674ac7ab8d9 admin admin] PATH_INFO: `/v3/projects/86d65ab5f7aa4fec9fcabf9264e4f819/users/c65fdbdb4a8d41b0aca274301c8fff1e/roles/9db5671c65d3414ea2682afa2ccf1403` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:18:08.954945 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1989f99f-397c-4636-acab-4e546afd7240 admin admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:18:08.957254 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-1989f99f-397c-4636-acab-4e546afd7240 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:18:08.980908 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef624b4-fa4a-4ee8-9070-7674ac7ab8d9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=86d65ab5f7aa4fec9fcabf9264e4f819, user_id=c65fdbdb4a8d41b0aca274301c8fff1e, role_id=9db5671c65d3414ea2682afa2ccf1403)` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:18:08.990776 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef624b4-fa4a-4ee8-9070-7674ac7ab8d9 admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:18:09.109262 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:18:09.117634 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 871/1725] 10.222.0.23 () {60 vars in 1482 bytes} [Thu May 19 12:18:08 2022] PUT /identity/v3/projects/86d65ab5f7aa4fec9fcabf9264e4f819/users/c65fdbdb4a8d41b0aca274301c8fff1e/roles/9db5671c65d3414ea2682afa2ccf1403 => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:18:09.123532 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef624b4-fa4a-4ee8-9070-7674ac7ab8d9 admin admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:18:09.317343 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1989f99f-397c-4636-acab-4e546afd7240 admin admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:18:09.452180 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef624b4-fa4a-4ee8-9070-7674ac7ab8d9 admin admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:18:09.486411 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4bcfb2-63af-4cce-83cb-a2e86c88970f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:18:09.494234 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4bcfb2-63af-4cce-83cb-a2e86c88970f admin admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:18:09.494533 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4bcfb2-63af-4cce-83cb-a2e86c88970f admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:18:09.494863 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4bcfb2-63af-4cce-83cb-a2e86c88970f admin admin] PATH_INFO: `/v3/roles` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:18:09.495695 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4bcfb2-63af-4cce-83cb-a2e86c88970f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:18:09.506081 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4bcfb2-63af-4cce-83cb-a2e86c88970f admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:18:09.545422 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 872/1726] 10.222.0.23 () {58 vars in 1121 bytes} [Thu May 19 12:18:09 2022] GET /identity/v3/roles => generated 1420 bytes in 421 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 12:18:09.555527 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4bcfb2-63af-4cce-83cb-a2e86c88970f admin admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:18:09.589475 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4bcfb2-63af-4cce-83cb-a2e86c88970f admin admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:18:09.608246 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3ad510-9bd1-4165-8e23-1dc974b8c2d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:18:09.609177 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3ad510-9bd1-4165-8e23-1dc974b8c2d5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:18:09.609865 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3ad510-9bd1-4165-8e23-1dc974b8c2d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:18:09.609865 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3ad510-9bd1-4165-8e23-1dc974b8c2d5 admin admin] PATH_INFO: `/v3/projects/86d65ab5f7aa4fec9fcabf9264e4f819/users/c65fdbdb4a8d41b0aca274301c8fff1e/roles/9db5671c65d3414ea2682afa2ccf1403` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:18:09.617418 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d49b53-5d4b-43ec-87bb-b54db33009dc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34a98d75d84b4c67981377e0f26eca62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd70d25bf69314820b8aaecb65576978a', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:18:09.626341 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d49b53-5d4b-43ec-87bb-b54db33009dc service nova] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:18:09.626341 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d49b53-5d4b-43ec-87bb-b54db33009dc service nova] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:18:09.626341 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d49b53-5d4b-43ec-87bb-b54db33009dc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:18:09.627232 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3ad510-9bd1-4165-8e23-1dc974b8c2d5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=86d65ab5f7aa4fec9fcabf9264e4f819, user_id=c65fdbdb4a8d41b0aca274301c8fff1e, role_id=9db5671c65d3414ea2682afa2ccf1403)` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:18:09.630216 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3ad510-9bd1-4165-8e23-1dc974b8c2d5 admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:18:09.677641 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 873/1727] 10.222.0.23 () {60 vars in 1482 bytes} [Thu May 19 12:18:09 2022] PUT /identity/v3/projects/86d65ab5f7aa4fec9fcabf9264e4f819/users/c65fdbdb4a8d41b0aca274301c8fff1e/roles/9db5671c65d3414ea2682afa2ccf1403 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:18:09.701209 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8baa46fa-b9f8-4525-8d82-d890c3f0fb04 None None] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:18:09.701548 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8baa46fa-b9f8-4525-8d82-d890c3f0fb04 None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:18:09.701836 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8baa46fa-b9f8-4525-8d82-d890c3f0fb04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:18:09.792727 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.auth.core [None req-8baa46fa-b9f8-4525-8d82-d890c3f0fb04 None None] MFA Rules not processed for user `c65fdbdb4a8d41b0aca274301c8fff1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:18:09.800682 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-8baa46fa-b9f8-4525-8d82-d890c3f0fb04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:18:09.865657 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d49b53-5d4b-43ec-87bb-b54db33009dc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:18:09.871703 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d49b53-5d4b-43ec-87bb-b54db33009dc service nova] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:18:09.910493 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 855/1728] 10.222.0.23 () {60 vars in 1365 bytes} [Thu May 19 12:18:08 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 956 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:18:10.064095 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d49b53-5d4b-43ec-87bb-b54db33009dc service nova] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:18:10.064095 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-a7d49b53-5d4b-43ec-87bb-b54db33009dc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:18:10.093545 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d49b53-5d4b-43ec-87bb-b54db33009dc service nova] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:18:10.114324 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:18:10.222386 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 874/1729] 10.222.0.23 () {58 vars in 957 bytes} [Thu May 19 12:18:09 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 522 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 12:18:10.253465 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8baa46fa-b9f8-4525-8d82-d890c3f0fb04 None None] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:18:10.268961 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-8baa46fa-b9f8-4525-8d82-d890c3f0fb04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:18:10.460980 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb41a2a-a086-498e-88e3-53c512149d97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:18:10.463364 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb41a2a-a086-498e-88e3-53c512149d97 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:18:10.467342 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb41a2a-a086-498e-88e3-53c512149d97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:18:10.467700 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8baa46fa-b9f8-4525-8d82-d890c3f0fb04 None None] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:18:10.467867 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb41a2a-a086-498e-88e3-53c512149d97 admin admin] PATH_INFO: `/v3/users/15a744f7a0f042258579fb0a3600f612` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:18:10.467867 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb41a2a-a086-498e-88e3-53c512149d97 admin admin] RBAC: Authorizing `identity:delete_user(user_id=15a744f7a0f042258579fb0a3600f612)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:18:10.470356 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb41a2a-a086-498e-88e3-53c512149d97 admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:18:10.475217 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133fadf4-621c-46ba-bb83-ac5e120d85bc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8800254c0e784e42a791fb9abc274d79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd70d25bf69314820b8aaecb65576978a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:18:10.477564 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133fadf4-621c-46ba-bb83-ac5e120d85bc service cinder] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:18:10.478337 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133fadf4-621c-46ba-bb83-ac5e120d85bc service cinder] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:18:10.478848 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133fadf4-621c-46ba-bb83-ac5e120d85bc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:18:10.544410 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-133fadf4-621c-46ba-bb83-ac5e120d85bc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:18:10.557588 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-133fadf4-621c-46ba-bb83-ac5e120d85bc service cinder] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:18:10.606531 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 875/1730] 10.222.0.23 () {60 vars in 1365 bytes} [Thu May 19 12:18:10 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:18:10.609314 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff110303-feaa-42d9-a3b8-f9aa74f8f624 None None] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:18:10.609678 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff110303-feaa-42d9-a3b8-f9aa74f8f624 None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:18:10.609926 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff110303-feaa-42d9-a3b8-f9aa74f8f624 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:18:10.670688 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.auth.core [None req-ff110303-feaa-42d9-a3b8-f9aa74f8f624 None None] MFA Rules not processed for user `1a00b3fa7497456895e5e0bf105c4fc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:18:10.673003 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-ff110303-feaa-42d9-a3b8-f9aa74f8f624 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:18:10.688235 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:18:10.848972 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-cfb41a2a-a086-498e-88e3-53c512149d97 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '15a744f7a0f042258579fb0a3600f612'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:18:10.879979 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 876/1731] 10.222.0.81 () {58 vars in 1015 bytes} [Thu May 19 12:18:10 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 272 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:18:10.891443 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-cfb41a2a-a086-498e-88e3-53c512149d97 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '15a744f7a0f042258579fb0a3600f612'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:18:10.892281 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-cfb41a2a-a086-498e-88e3-53c512149d97 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '15a744f7a0f042258579fb0a3600f612'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:18:10.995377 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-cfb41a2a-a086-498e-88e3-53c512149d97 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '15a744f7a0f042258579fb0a3600f612'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:18:11.007239 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-cfb41a2a-a086-498e-88e3-53c512149d97 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '15a744f7a0f042258579fb0a3600f612'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:18:11.046754 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 856/1732] 10.222.0.23 () {58 vars in 1223 bytes} [Thu May 19 12:18:10 2022] DELETE /identity/v3/users/15a744f7a0f042258579fb0a3600f612 => generated 0 bytes in 991 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 12:18:11.052255 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794ca579-ff56-44c7-88c2-1c03914ddaca None None] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:18:11.052798 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794ca579-ff56-44c7-88c2-1c03914ddaca None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:18:11.053313 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794ca579-ff56-44c7-88c2-1c03914ddaca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:18:11.095387 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.auth.core [None req-794ca579-ff56-44c7-88c2-1c03914ddaca None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:18:11.097641 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-794ca579-ff56-44c7-88c2-1c03914ddaca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:18:11.319078 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb41a2a-a086-498e-88e3-53c512149d97 admin admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:18:11.322400 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-cfb41a2a-a086-498e-88e3-53c512149d97 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:18:11.507103 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 877/1733] 10.222.0.23 () {58 vars in 957 bytes} [Thu May 19 12:18:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 456 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:18:11.798639 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb41a2a-a086-498e-88e3-53c512149d97 admin admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:18:11.808079 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ee29ae-a6aa-468e-8272-1c1c834d7301 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a00b3fa7497456895e5e0bf105c4fc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd70d25bf69314820b8aaecb65576978a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:18:11.808769 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ee29ae-a6aa-468e-8272-1c1c834d7301 service neutron] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:18:11.809090 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ee29ae-a6aa-468e-8272-1c1c834d7301 service neutron] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:18:11.809377 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ee29ae-a6aa-468e-8272-1c1c834d7301 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:18:11.814163 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-99ee29ae-a6aa-468e-8272-1c1c834d7301 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:18:11.831378 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794ca579-ff56-44c7-88c2-1c03914ddaca None None] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:18:11.835958 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-794ca579-ff56-44c7-88c2-1c03914ddaca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:18:11.871629 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794ca579-ff56-44c7-88c2-1c03914ddaca None None] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:18:11.881528 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ee29ae-a6aa-468e-8272-1c1c834d7301 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:18:11.881528 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ee29ae-a6aa-468e-8272-1c1c834d7301 service neutron] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:18:11.891797 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85729190-149b-49b1-99f7-49520bba1fb0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a00b3fa7497456895e5e0bf105c4fc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd70d25bf69314820b8aaecb65576978a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:18:11.909464 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85729190-149b-49b1-99f7-49520bba1fb0 service neutron] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:18:11.913387 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85729190-149b-49b1-99f7-49520bba1fb0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:18:11.916215 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85729190-149b-49b1-99f7-49520bba1fb0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:18:11.949756 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 857/1734] 10.222.0.23 () {60 vars in 1365 bytes} [Thu May 19 12:18:11 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 631 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 12:18:11.966336 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85729190-149b-49b1-99f7-49520bba1fb0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:18:11.973220 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85729190-149b-49b1-99f7-49520bba1fb0 service neutron] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:18:11.998559 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 878/1735] 10.222.0.23 () {60 vars in 1365 bytes} [Thu May 19 12:18:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 12:18:12.434985 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d2d5af-ab08-4dcb-b2da-47bfe11ca046 None None] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:18:12.435670 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d2d5af-ab08-4dcb-b2da-47bfe11ca046 None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:18:12.436048 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d2d5af-ab08-4dcb-b2da-47bfe11ca046 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:18:12.563969 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.auth.core [None req-f1d2d5af-ab08-4dcb-b2da-47bfe11ca046 None None] MFA Rules not processed for user `dd39ddcf4ab045b98a737ee529a2dcce`. Rule list: `[]` (Enabled: `True`). {{(pid=80539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:18:12.565469 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-f1d2d5af-ab08-4dcb-b2da-47bfe11ca046 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:18:12.735907 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:18:13.254012 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 858/1736] 10.222.0.23 () {58 vars in 957 bytes} [Thu May 19 12:18:12 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 813 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 12:18:13.286512 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85729190-149b-49b1-99f7-49520bba1fb0 service neutron] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:18:13.292295 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-85729190-149b-49b1-99f7-49520bba1fb0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:18:13.707820 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85729190-149b-49b1-99f7-49520bba1fb0 service neutron] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:18:13.747933 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b657d8d-56e8-4d7f-813b-e5b761e032da service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34a98d75d84b4c67981377e0f26eca62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd70d25bf69314820b8aaecb65576978a', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:18:13.748641 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b657d8d-56e8-4d7f-813b-e5b761e032da service nova] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:18:13.748936 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b657d8d-56e8-4d7f-813b-e5b761e032da service nova] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:18:13.749223 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b657d8d-56e8-4d7f-813b-e5b761e032da service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:18:13.801943 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b657d8d-56e8-4d7f-813b-e5b761e032da service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:18:13.803723 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b657d8d-56e8-4d7f-813b-e5b761e032da service nova] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:18:13.841232 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d2d5af-ab08-4dcb-b2da-47bfe11ca046 None None] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:18:13.844126 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-f1d2d5af-ab08-4dcb-b2da-47bfe11ca046 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:18:13.856850 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 879/1737] 10.222.0.23 () {60 vars in 1365 bytes} [Thu May 19 12:18:13 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 572 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:18:14.009337 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d2d5af-ab08-4dcb-b2da-47bfe11ca046 None None] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:18:14.046458 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead41d6b-237c-4d91-ac46-942058a03a00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:18:14.063366 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead41d6b-237c-4d91-ac46-942058a03a00 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:18:14.064522 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead41d6b-237c-4d91-ac46-942058a03a00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:18:14.067975 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead41d6b-237c-4d91-ac46-942058a03a00 admin admin] PATH_INFO: `/v3/projects/65edf999e3b24c29815d321084d62bca` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:18:14.077580 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead41d6b-237c-4d91-ac46-942058a03a00 admin admin] RBAC: Authorizing `identity:delete_project(project_id=65edf999e3b24c29815d321084d62bca)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:18:14.094520 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead41d6b-237c-4d91-ac46-942058a03a00 admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:18:14.470152 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-ead41d6b-237c-4d91-ac46-942058a03a00 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '65edf999e3b24c29815d321084d62bca', 'request_id': 'req-ead41d6b-237c-4d91-ac46-942058a03a00'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:18:14.504105 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-ead41d6b-237c-4d91-ac46-942058a03a00 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '65edf999e3b24c29815d321084d62bca', 'request_id': 'req-ead41d6b-237c-4d91-ac46-942058a03a00'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:18:14.663174 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-ead41d6b-237c-4d91-ac46-942058a03a00 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '65edf999e3b24c29815d321084d62bca', 'request_id': 'req-ead41d6b-237c-4d91-ac46-942058a03a00'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:18:14.723938 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-ead41d6b-237c-4d91-ac46-942058a03a00 admin admin] The token cache is being invalidate because project 65edf999e3b24c29815d321084d62bca was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80539) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 12:18:14.729718 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-ead41d6b-237c-4d91-ac46-942058a03a00 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:18:14.733575 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-ead41d6b-237c-4d91-ac46-942058a03a00 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:18:14.749838 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 859/1738] 10.222.0.23 () {58 vars in 1232 bytes} [Thu May 19 12:18:13 2022] DELETE /identity/v3/projects/65edf999e3b24c29815d321084d62bca => generated 0 bytes in 904 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:18:14.750126 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b657d8d-56e8-4d7f-813b-e5b761e032da service nova] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:18:14.770542 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b657d8d-56e8-4d7f-813b-e5b761e032da service nova] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:18:14.875534 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:18:14.998256 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b524d0c2-9fc7-4393-8713-872f599e7e92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:18:14.999465 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b524d0c2-9fc7-4393-8713-872f599e7e92 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:18:15.000012 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b524d0c2-9fc7-4393-8713-872f599e7e92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:18:15.000443 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b524d0c2-9fc7-4393-8713-872f599e7e92 admin admin] PATH_INFO: `/v3/users/798f925e652944e089359754e0181e7a` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:18:15.004963 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b524d0c2-9fc7-4393-8713-872f599e7e92 admin admin] RBAC: Authorizing `identity:delete_user(user_id=798f925e652944e089359754e0181e7a)` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:18:15.016185 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b524d0c2-9fc7-4393-8713-872f599e7e92 admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:18:15.334915 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-b524d0c2-9fc7-4393-8713-872f599e7e92 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '798f925e652944e089359754e0181e7a'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:18:15.395394 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-b524d0c2-9fc7-4393-8713-872f599e7e92 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '798f925e652944e089359754e0181e7a'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:18:15.401979 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-b524d0c2-9fc7-4393-8713-872f599e7e92 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '798f925e652944e089359754e0181e7a'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:18:15.503253 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-b524d0c2-9fc7-4393-8713-872f599e7e92 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '798f925e652944e089359754e0181e7a'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:18:15.535994 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-b524d0c2-9fc7-4393-8713-872f599e7e92 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '798f925e652944e089359754e0181e7a'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:18:15.603603 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 880/1739] 10.222.0.23 () {58 vars in 1223 bytes} [Thu May 19 12:18:14 2022] DELETE /identity/v3/users/798f925e652944e089359754e0181e7a => generated 0 bytes in 855 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:18:16.236512 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482aa287-beab-4140-99df-2e716001399a None None] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:18:16.237976 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482aa287-beab-4140-99df-2e716001399a None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:18:16.244943 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482aa287-beab-4140-99df-2e716001399a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:18:16.340681 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.auth.core [None req-482aa287-beab-4140-99df-2e716001399a None None] MFA Rules not processed for user `8c99000c8e6f4a98b430bfd479f87287`. Rule list: `[]` (Enabled: `True`). {{(pid=80539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:18:16.349993 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-482aa287-beab-4140-99df-2e716001399a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:18:16.665050 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 860/1740] 10.222.0.23 () {58 vars in 957 bytes} [Thu May 19 12:18:16 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 427 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:18:16.673967 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b524d0c2-9fc7-4393-8713-872f599e7e92 admin admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:18:16.675431 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-b524d0c2-9fc7-4393-8713-872f599e7e92 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:18:16.913949 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b524d0c2-9fc7-4393-8713-872f599e7e92 admin admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:18:16.930531 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482aa287-beab-4140-99df-2e716001399a None None] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:18:16.934079 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-482aa287-beab-4140-99df-2e716001399a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:18:16.941431 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb7e5c3-15cd-4102-9a21-5cedbfc3c3b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:18:16.960949 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb7e5c3-15cd-4102-9a21-5cedbfc3c3b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:18:16.960949 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb7e5c3-15cd-4102-9a21-5cedbfc3c3b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:18:16.960949 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb7e5c3-15cd-4102-9a21-5cedbfc3c3b1 admin admin] PATH_INFO: `/v3/projects/4c4d3055c4f544ada2d283b6f8e95405` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:18:16.964620 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb7e5c3-15cd-4102-9a21-5cedbfc3c3b1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4c4d3055c4f544ada2d283b6f8e95405)` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:18:16.971512 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb7e5c3-15cd-4102-9a21-5cedbfc3c3b1 admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:18:16.983669 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482aa287-beab-4140-99df-2e716001399a None None] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:18:16.996571 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a35d854-77e4-4ce4-8208-5438e0717644 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:18:16.997893 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a35d854-77e4-4ce4-8208-5438e0717644 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:18:16.998967 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a35d854-77e4-4ce4-8208-5438e0717644 admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:18:16.999261 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a35d854-77e4-4ce4-8208-5438e0717644 admin admin] PATH_INFO: `/v3/users/4334c63b156e44908453a320dbe0524b` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:18:17.007258 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a35d854-77e4-4ce4-8208-5438e0717644 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4334c63b156e44908453a320dbe0524b)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:18:17.010529 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a35d854-77e4-4ce4-8208-5438e0717644 admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:18:17.059626 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:18:17.368545 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-dcb7e5c3-15cd-4102-9a21-5cedbfc3c3b1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4c4d3055c4f544ada2d283b6f8e95405', 'request_id': 'req-dcb7e5c3-15cd-4102-9a21-5cedbfc3c3b1'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:18:17.385059 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:18:17.403599 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-2a35d854-77e4-4ce4-8208-5438e0717644 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4334c63b156e44908453a320dbe0524b'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:18:17.423674 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-dcb7e5c3-15cd-4102-9a21-5cedbfc3c3b1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4c4d3055c4f544ada2d283b6f8e95405', 'request_id': 'req-dcb7e5c3-15cd-4102-9a21-5cedbfc3c3b1'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:18:17.470360 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-dcb7e5c3-15cd-4102-9a21-5cedbfc3c3b1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4c4d3055c4f544ada2d283b6f8e95405', 'request_id': 'req-dcb7e5c3-15cd-4102-9a21-5cedbfc3c3b1'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:18:17.490628 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-2a35d854-77e4-4ce4-8208-5438e0717644 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4334c63b156e44908453a320dbe0524b'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:18:17.502337 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-2a35d854-77e4-4ce4-8208-5438e0717644 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4334c63b156e44908453a320dbe0524b'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:18:17.668342 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-dcb7e5c3-15cd-4102-9a21-5cedbfc3c3b1 admin admin] The token cache is being invalidate because project 4c4d3055c4f544ada2d283b6f8e95405 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80540) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 12:18:17.670150 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-dcb7e5c3-15cd-4102-9a21-5cedbfc3c3b1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:18:17.672431 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-dcb7e5c3-15cd-4102-9a21-5cedbfc3c3b1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:18:17.690334 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 881/1741] 10.222.0.23 () {58 vars in 1232 bytes} [Thu May 19 12:18:16 2022] DELETE /identity/v3/projects/4c4d3055c4f544ada2d283b6f8e95405 => generated 0 bytes in 1014 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:18:17.699333 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17091077-2cd4-411f-8736-3ce1f33231f1 None None] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:18:17.700523 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17091077-2cd4-411f-8736-3ce1f33231f1 None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:18:17.701331 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17091077-2cd4-411f-8736-3ce1f33231f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:18:17.712683 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-2a35d854-77e4-4ce4-8208-5438e0717644 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4334c63b156e44908453a320dbe0524b'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:18:17.741687 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-2a35d854-77e4-4ce4-8208-5438e0717644 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4334c63b156e44908453a320dbe0524b'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:18:17.753177 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.auth.core [None req-17091077-2cd4-411f-8736-3ce1f33231f1 None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:18:17.768353 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-17091077-2cd4-411f-8736-3ce1f33231f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:18:17.806606 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 861/1742] 10.222.0.23 () {58 vars in 1223 bytes} [Thu May 19 12:18:16 2022] DELETE /identity/v3/users/4334c63b156e44908453a320dbe0524b => generated 0 bytes in 877 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 12:18:17.813169 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1041d70-5cf9-4bcb-9035-d46d4bdc17fc None None] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:18:17.813475 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1041d70-5cf9-4bcb-9035-d46d4bdc17fc None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:18:17.813765 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1041d70-5cf9-4bcb-9035-d46d4bdc17fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:18:17.874651 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.auth.core [None req-c1041d70-5cf9-4bcb-9035-d46d4bdc17fc None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:18:17.880005 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-c1041d70-5cf9-4bcb-9035-d46d4bdc17fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:18:18.013949 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 882/1743] 10.222.0.23 () {58 vars in 957 bytes} [Thu May 19 12:18:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 321 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:18:18.028294 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf87c515-6d2d-493a-8011-843efd38db02 None None] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:18:18.034172 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf87c515-6d2d-493a-8011-843efd38db02 None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:18:18.034172 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf87c515-6d2d-493a-8011-843efd38db02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:18:18.124252 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.auth.core [None req-cf87c515-6d2d-493a-8011-843efd38db02 None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:18:18.134101 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-cf87c515-6d2d-493a-8011-843efd38db02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:18:18.190400 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 862/1744] 10.222.0.23 () {58 vars in 957 bytes} [Thu May 19 12:18:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 378 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 12:18:18.202313 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6190ad-f78b-4d45-a4df-8c5bc4be70e4 None None] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:18:18.203418 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6190ad-f78b-4d45-a4df-8c5bc4be70e4 None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:18:18.205589 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6190ad-f78b-4d45-a4df-8c5bc4be70e4 None None] PATH_INFO: `/` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:18:18.215407 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 863/1745] 10.222.0.23 () {54 vars in 921 bytes} [Thu May 19 12:18:18 2022] GET /identity => generated 270 bytes in 24 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 12:18:18.276583 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a746dee9-a526-4ac2-a995-5e9bb232fc2c None None] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:18:18.277993 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a746dee9-a526-4ac2-a995-5e9bb232fc2c None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:18:18.279014 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a746dee9-a526-4ac2-a995-5e9bb232fc2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:18:18.361383 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.auth.core [None req-a746dee9-a526-4ac2-a995-5e9bb232fc2c None None] MFA Rules not processed for user `34a98d75d84b4c67981377e0f26eca62`. Rule list: `[]` (Enabled: `True`). {{(pid=80539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:18:18.368136 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-a746dee9-a526-4ac2-a995-5e9bb232fc2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:18:18.458742 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 883/1746] 10.222.0.23 () {58 vars in 957 bytes} [Thu May 19 12:18:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 430 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:18:18.462210 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf87c515-6d2d-493a-8011-843efd38db02 None None] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:18:18.468934 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-cf87c515-6d2d-493a-8011-843efd38db02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:18:18.660010 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 864/1747] 10.222.0.23 () {58 vars in 1018 bytes} [Thu May 19 12:18:18 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 385 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:18:18.662063 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a746dee9-a526-4ac2-a995-5e9bb232fc2c None None] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:18:18.684527 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a746dee9-a526-4ac2-a995-5e9bb232fc2c None None] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:18:18.695756 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1741ddcd-b3cd-462d-80c4-378960f606e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:18:18.696816 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1741ddcd-b3cd-462d-80c4-378960f606e1 admin admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:18:18.697315 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1741ddcd-b3cd-462d-80c4-378960f606e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:18:18.697794 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1741ddcd-b3cd-462d-80c4-378960f606e1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:18:18.698800 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1741ddcd-b3cd-462d-80c4-378960f606e1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:18:18.699318 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1741ddcd-b3cd-462d-80c4-378960f606e1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:18:18.701398 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1741ddcd-b3cd-462d-80c4-378960f606e1 admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:18:18.735228 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 865/1748] 10.222.0.23 () {58 vars in 1152 bytes} [Thu May 19 12:18:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:18:18.748895 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf87c515-6d2d-493a-8011-843efd38db02 None None] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:18:18.753556 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b45a8b9-53e9-4af6-8a0c-da84cf3b7129 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a00b3fa7497456895e5e0bf105c4fc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd70d25bf69314820b8aaecb65576978a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:18:18.754557 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b45a8b9-53e9-4af6-8a0c-da84cf3b7129 service neutron] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:18:18.754989 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b45a8b9-53e9-4af6-8a0c-da84cf3b7129 service neutron] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:18:18.755445 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b45a8b9-53e9-4af6-8a0c-da84cf3b7129 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:18:18.762364 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1741ddcd-b3cd-462d-80c4-378960f606e1 admin admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:18:18.774527 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b45a8b9-53e9-4af6-8a0c-da84cf3b7129 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:18:18.776570 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b45a8b9-53e9-4af6-8a0c-da84cf3b7129 service neutron] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:18:18.819238 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 884/1749] 10.222.0.23 () {60 vars in 1365 bytes} [Thu May 19 12:18:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:18:18.821766 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b45a8b9-53e9-4af6-8a0c-da84cf3b7129 service neutron] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:18:18.823642 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1741ddcd-b3cd-462d-80c4-378960f606e1 admin admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:18:18.824369 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-8b45a8b9-53e9-4af6-8a0c-da84cf3b7129 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:18:18.844635 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2509f41c-3396-47ad-bfbf-cc57037c1b28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:18:18.848592 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2509f41c-3396-47ad-bfbf-cc57037c1b28 admin admin] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:18:18.854027 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2509f41c-3396-47ad-bfbf-cc57037c1b28 admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:18:18.855001 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2509f41c-3396-47ad-bfbf-cc57037c1b28 admin admin] PATH_INFO: `/v3/projects` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:18:18.855158 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b45a8b9-53e9-4af6-8a0c-da84cf3b7129 service neutron] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:18:18.856837 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2509f41c-3396-47ad-bfbf-cc57037c1b28 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:18:18.860866 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2509f41c-3396-47ad-bfbf-cc57037c1b28 admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:18:18.872324 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7913a1e9-7daf-4413-9ba2-b90fd9f67da3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34a98d75d84b4c67981377e0f26eca62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd70d25bf69314820b8aaecb65576978a', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:18:18.873120 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7913a1e9-7daf-4413-9ba2-b90fd9f67da3 service nova] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:18:18.873409 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7913a1e9-7daf-4413-9ba2-b90fd9f67da3 service nova] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:18:18.873707 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7913a1e9-7daf-4413-9ba2-b90fd9f67da3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:18:18.887632 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7913a1e9-7daf-4413-9ba2-b90fd9f67da3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:18:18.888926 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7913a1e9-7daf-4413-9ba2-b90fd9f67da3 service nova] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:18:18.905891 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 885/1750] 10.222.0.23 () {60 vars in 1365 bytes} [Thu May 19 12:18:18 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:18:19.022409 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=92,keystone:DELETE=6,keystone:INSERT=2 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:18:19.037109 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 866/1751] 10.222.0.23 () {60 vars in 1152 bytes} [Thu May 19 12:18:18 2022] POST /identity/v3/projects => generated 365 bytes in 276 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 12:18:19.041643 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7913a1e9-7daf-4413-9ba2-b90fd9f67da3 service nova] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:18:19.060886 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7913a1e9-7daf-4413-9ba2-b90fd9f67da3 service nova] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:18:19.083460 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=96,keystone:DELETE=3,keystone:INSERT=1 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:18:19.269709 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9652a2d3-7b18-4d55-a59c-914a8a660789 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:18:19.271526 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9652a2d3-7b18-4d55-a59c-914a8a660789 admin admin] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:18:19.272936 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9652a2d3-7b18-4d55-a59c-914a8a660789 admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:18:19.274044 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9652a2d3-7b18-4d55-a59c-914a8a660789 admin admin] PATH_INFO: `/v3/users` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:18:19.275712 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9652a2d3-7b18-4d55-a59c-914a8a660789 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:18:19.280141 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9652a2d3-7b18-4d55-a59c-914a8a660789 admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:18:19.443304 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 886/1752] 10.222.0.23 () {60 vars in 1143 bytes} [Thu May 19 12:18:19 2022] POST /identity/v3/users => generated 283 bytes in 403 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 12:18:19.449808 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2509f41c-3396-47ad-bfbf-cc57037c1b28 admin admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:18:19.518266 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2509f41c-3396-47ad-bfbf-cc57037c1b28 admin admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:18:19.551171 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5c1d66-6278-4b3b-acee-07acdfae67e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:18:19.558015 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5c1d66-6278-4b3b-acee-07acdfae67e1 admin admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:18:19.558386 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5c1d66-6278-4b3b-acee-07acdfae67e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:18:19.558684 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5c1d66-6278-4b3b-acee-07acdfae67e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:18:19.560746 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5c1d66-6278-4b3b-acee-07acdfae67e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:18:19.565247 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5c1d66-6278-4b3b-acee-07acdfae67e1 admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:18:19.610076 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 867/1753] 10.222.0.23 () {58 vars in 1121 bytes} [Thu May 19 12:18:19 2022] GET /identity/v3/roles => generated 1420 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 12:18:19.611613 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9652a2d3-7b18-4d55-a59c-914a8a660789 admin admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:18:19.648038 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9652a2d3-7b18-4d55-a59c-914a8a660789 admin admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:18:19.668324 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9531a63d-fbdb-4815-90d8-0a81091ea70a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:18:19.669282 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9531a63d-fbdb-4815-90d8-0a81091ea70a admin admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:18:19.669792 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9531a63d-fbdb-4815-90d8-0a81091ea70a admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:18:19.670286 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9531a63d-fbdb-4815-90d8-0a81091ea70a admin admin] PATH_INFO: `/v3/roles` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:18:19.673186 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9531a63d-fbdb-4815-90d8-0a81091ea70a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:18:19.689560 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9531a63d-fbdb-4815-90d8-0a81091ea70a admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:18:19.719860 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 887/1754] 10.222.0.23 () {58 vars in 1121 bytes} [Thu May 19 12:18:19 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 19 12:18:19.723403 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5c1d66-6278-4b3b-acee-07acdfae67e1 admin admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:18:19.740468 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5c1d66-6278-4b3b-acee-07acdfae67e1 admin admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:18:19.750343 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc7a93f-7bf9-470e-80f2-9761eb15571b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:18:19.751607 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc7a93f-7bf9-470e-80f2-9761eb15571b admin admin] REQUEST_METHOD: `PUT` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:18:19.752110 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc7a93f-7bf9-470e-80f2-9761eb15571b admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:18:19.752567 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc7a93f-7bf9-470e-80f2-9761eb15571b admin admin] PATH_INFO: `/v3/projects/3f82f9f8661b44709371c4d574b4de0a/users/e6367f23eb0c4c1e88fd349c4d3fc073/roles/259eb400cdb4439db72d14456ae9cd08` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:18:19.795579 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc7a93f-7bf9-470e-80f2-9761eb15571b admin admin] RBAC: Authorizing `identity:create_grant(project_id=3f82f9f8661b44709371c4d574b4de0a, user_id=e6367f23eb0c4c1e88fd349c4d3fc073, role_id=259eb400cdb4439db72d14456ae9cd08)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:18:19.803026 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc7a93f-7bf9-470e-80f2-9761eb15571b admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:18:19.965486 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 868/1755] 10.222.0.23 () {60 vars in 1482 bytes} [Thu May 19 12:18:19 2022] PUT /identity/v3/projects/3f82f9f8661b44709371c4d574b4de0a/users/e6367f23eb0c4c1e88fd349c4d3fc073/roles/259eb400cdb4439db72d14456ae9cd08 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:18:19.968788 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52a1e3a-1cc8-4b46-bd59-609f51154155 None None] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:18:19.969472 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52a1e3a-1cc8-4b46-bd59-609f51154155 None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:18:19.970066 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52a1e3a-1cc8-4b46-bd59-609f51154155 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:18:20.053499 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.auth.core [None req-d52a1e3a-1cc8-4b46-bd59-609f51154155 None None] MFA Rules not processed for user `e6367f23eb0c4c1e88fd349c4d3fc073`. Rule list: `[]` (Enabled: `True`). {{(pid=80540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:18:20.061168 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-d52a1e3a-1cc8-4b46-bd59-609f51154155 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:18:20.303035 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 888/1756] 10.222.0.23 () {58 vars in 957 bytes} [Thu May 19 12:18:19 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 336 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:18:20.309521 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc7a93f-7bf9-470e-80f2-9761eb15571b admin admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:18:20.462222 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc7a93f-7bf9-470e-80f2-9761eb15571b admin admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:18:20.471028 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cafc4fe-332e-4bef-87b7-dbe361567f2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:18:20.472481 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cafc4fe-332e-4bef-87b7-dbe361567f2b admin admin] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:18:20.472971 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cafc4fe-332e-4bef-87b7-dbe361567f2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:18:20.473695 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cafc4fe-332e-4bef-87b7-dbe361567f2b admin admin] PATH_INFO: `/v3/projects` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:18:20.474639 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cafc4fe-332e-4bef-87b7-dbe361567f2b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:18:20.478457 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cafc4fe-332e-4bef-87b7-dbe361567f2b admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:18:20.666562 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 869/1757] 10.222.0.23 () {60 vars in 1152 bytes} [Thu May 19 12:18:20 2022] POST /identity/v3/projects => generated 361 bytes in 358 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 12:18:20.672454 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52a1e3a-1cc8-4b46-bd59-609f51154155 None None] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:18:20.714626 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52a1e3a-1cc8-4b46-bd59-609f51154155 None None] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:18:21.065236 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6cdc347-79d2-4d44-915a-06a193b72295 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:18:21.066335 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cdc347-79d2-4d44-915a-06a193b72295 admin admin] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:18:21.066790 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cdc347-79d2-4d44-915a-06a193b72295 admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:18:21.067165 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cdc347-79d2-4d44-915a-06a193b72295 admin admin] PATH_INFO: `/v3/users` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:18:21.068381 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6cdc347-79d2-4d44-915a-06a193b72295 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:18:21.071725 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6cdc347-79d2-4d44-915a-06a193b72295 admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:18:21.220030 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 889/1758] 10.222.0.23 () {60 vars in 1143 bytes} [Thu May 19 12:18:20 2022] POST /identity/v3/users => generated 287 bytes in 545 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 12:18:21.222333 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cafc4fe-332e-4bef-87b7-dbe361567f2b admin admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:18:21.243461 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cafc4fe-332e-4bef-87b7-dbe361567f2b admin admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:18:21.257901 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9f7747-51a0-4950-9484-8cb1752cf5dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:18:21.258872 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9f7747-51a0-4950-9484-8cb1752cf5dc admin admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:18:21.259179 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9f7747-51a0-4950-9484-8cb1752cf5dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:18:21.259461 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9f7747-51a0-4950-9484-8cb1752cf5dc admin admin] PATH_INFO: `/v3/roles` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:18:21.267294 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9f7747-51a0-4950-9484-8cb1752cf5dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:18:21.269385 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9f7747-51a0-4950-9484-8cb1752cf5dc admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:18:21.275181 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6cdc347-79d2-4d44-915a-06a193b72295 admin admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:18:21.277375 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-a6cdc347-79d2-4d44-915a-06a193b72295 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:18:21.318608 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 870/1759] 10.222.0.23 () {58 vars in 1121 bytes} [Thu May 19 12:18:21 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 12:18:21.318859 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6cdc347-79d2-4d44-915a-06a193b72295 admin admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:18:21.322191 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9f7747-51a0-4950-9484-8cb1752cf5dc admin admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:18:21.340969 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5d5f91-de58-48b0-8198-7a4abc470533 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:18:21.345775 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5d5f91-de58-48b0-8198-7a4abc470533 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:18:21.347808 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5d5f91-de58-48b0-8198-7a4abc470533 admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:18:21.349293 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5d5f91-de58-48b0-8198-7a4abc470533 admin admin] PATH_INFO: `/v3/projects/d5a205482a6c4dd8b18cd749bc2e45ff` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:18:21.359767 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5d5f91-de58-48b0-8198-7a4abc470533 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d5a205482a6c4dd8b18cd749bc2e45ff)` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:18:21.360655 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9f7747-51a0-4950-9484-8cb1752cf5dc admin admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:18:21.369671 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9b916c-af74-49cd-8c46-7f255d03813f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:18:21.371152 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9b916c-af74-49cd-8c46-7f255d03813f admin admin] REQUEST_METHOD: `PUT` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:18:21.371445 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9b916c-af74-49cd-8c46-7f255d03813f admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:18:21.371729 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9b916c-af74-49cd-8c46-7f255d03813f admin admin] PATH_INFO: `/v3/projects/bdb86b1eb0cc48eeab60c78e68cb510f/users/4174b35cc2ee4dd880b1e978de360413/roles/9db5671c65d3414ea2682afa2ccf1403` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:18:21.374682 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5d5f91-de58-48b0-8198-7a4abc470533 admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:18:21.387165 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:18:21.397207 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9b916c-af74-49cd-8c46-7f255d03813f admin admin] RBAC: Authorizing `identity:create_grant(project_id=bdb86b1eb0cc48eeab60c78e68cb510f, user_id=4174b35cc2ee4dd880b1e978de360413, role_id=9db5671c65d3414ea2682afa2ccf1403)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:18:21.403218 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9b916c-af74-49cd-8c46-7f255d03813f admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:18:21.543229 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 871/1760] 10.222.0.23 () {60 vars in 1482 bytes} [Thu May 19 12:18:21 2022] PUT /identity/v3/projects/bdb86b1eb0cc48eeab60c78e68cb510f/users/4174b35cc2ee4dd880b1e978de360413/roles/9db5671c65d3414ea2682afa2ccf1403 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:18:21.549556 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9b916c-af74-49cd-8c46-7f255d03813f admin admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:18:21.725576 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-8c5d5f91-de58-48b0-8198-7a4abc470533 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd5a205482a6c4dd8b18cd749bc2e45ff', 'request_id': 'req-8c5d5f91-de58-48b0-8198-7a4abc470533'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:18:21.771820 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-8c5d5f91-de58-48b0-8198-7a4abc470533 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd5a205482a6c4dd8b18cd749bc2e45ff', 'request_id': 'req-8c5d5f91-de58-48b0-8198-7a4abc470533'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:18:21.801163 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-8c5d5f91-de58-48b0-8198-7a4abc470533 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd5a205482a6c4dd8b18cd749bc2e45ff', 'request_id': 'req-8c5d5f91-de58-48b0-8198-7a4abc470533'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:18:21.845059 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9b916c-af74-49cd-8c46-7f255d03813f admin admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:18:21.903667 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-8c5d5f91-de58-48b0-8198-7a4abc470533 admin admin] The token cache is being invalidate because project d5a205482a6c4dd8b18cd749bc2e45ff was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80540) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 12:18:21.904047 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-8c5d5f91-de58-48b0-8198-7a4abc470533 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:18:21.905316 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-8c5d5f91-de58-48b0-8198-7a4abc470533 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:18:21.914258 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 890/1761] 10.222.0.23 () {58 vars in 1232 bytes} [Thu May 19 12:18:21 2022] DELETE /identity/v3/projects/d5a205482a6c4dd8b18cd749bc2e45ff => generated 0 bytes in 634 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:18:21.917025 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7c5d4d-0189-4f5b-a679-ac1cf4b57b5a None None] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:18:21.917371 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7c5d4d-0189-4f5b-a679-ac1cf4b57b5a None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:18:21.917649 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7c5d4d-0189-4f5b-a679-ac1cf4b57b5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:18:21.923087 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:18:21.992593 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.auth.core [None req-6a7c5d4d-0189-4f5b-a679-ac1cf4b57b5a None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:18:21.997267 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-6a7c5d4d-0189-4f5b-a679-ac1cf4b57b5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:18:22.075052 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68d3fe5-386a-40e8-b1af-09646cf11c5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:18:22.078376 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68d3fe5-386a-40e8-b1af-09646cf11c5f admin admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:18:22.078920 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68d3fe5-386a-40e8-b1af-09646cf11c5f admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:18:22.079185 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68d3fe5-386a-40e8-b1af-09646cf11c5f admin admin] PATH_INFO: `/v3/roles` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:18:22.080032 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68d3fe5-386a-40e8-b1af-09646cf11c5f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:18:22.092419 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68d3fe5-386a-40e8-b1af-09646cf11c5f admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:18:22.157412 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 872/1762] 10.222.0.23 () {58 vars in 1121 bytes} [Thu May 19 12:18:21 2022] GET /identity/v3/roles => generated 1420 bytes in 605 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 12:18:22.157816 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68d3fe5-386a-40e8-b1af-09646cf11c5f admin admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:18:22.165238 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-e68d3fe5-386a-40e8-b1af-09646cf11c5f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:18:22.193077 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68d3fe5-386a-40e8-b1af-09646cf11c5f admin admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:18:22.208325 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5013a1f2-be51-46f5-bf09-15a43e536c97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:18:22.211063 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5013a1f2-be51-46f5-bf09-15a43e536c97 admin admin] REQUEST_METHOD: `PUT` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:18:22.211358 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5013a1f2-be51-46f5-bf09-15a43e536c97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:18:22.211829 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5013a1f2-be51-46f5-bf09-15a43e536c97 admin admin] PATH_INFO: `/v3/projects/bdb86b1eb0cc48eeab60c78e68cb510f/users/4174b35cc2ee4dd880b1e978de360413/roles/9db5671c65d3414ea2682afa2ccf1403` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:18:22.218311 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5013a1f2-be51-46f5-bf09-15a43e536c97 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bdb86b1eb0cc48eeab60c78e68cb510f, user_id=4174b35cc2ee4dd880b1e978de360413, role_id=9db5671c65d3414ea2682afa2ccf1403)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:18:22.219483 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 891/1763] 10.222.0.23 () {58 vars in 957 bytes} [Thu May 19 12:18:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 304 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 12:18:22.221578 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5013a1f2-be51-46f5-bf09-15a43e536c97 admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:18:22.230725 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19edae93-533d-40b6-8862-8bd3871094d9 None None] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:18:22.231185 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19edae93-533d-40b6-8862-8bd3871094d9 None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:18:22.231474 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19edae93-533d-40b6-8862-8bd3871094d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:18:22.265120 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 873/1764] 10.222.0.23 () {60 vars in 1482 bytes} [Thu May 19 12:18:22 2022] PUT /identity/v3/projects/bdb86b1eb0cc48eeab60c78e68cb510f/users/4174b35cc2ee4dd880b1e978de360413/roles/9db5671c65d3414ea2682afa2ccf1403 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 12:18:22.286448 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2381e9b3-ea99-4c19-ab89-fdaa49ded91b None None] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:18:22.293825 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2381e9b3-ea99-4c19-ab89-fdaa49ded91b None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:18:22.295141 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2381e9b3-ea99-4c19-ab89-fdaa49ded91b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:18:22.307214 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.auth.core [None req-19edae93-533d-40b6-8862-8bd3871094d9 None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:18:22.310802 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-19edae93-533d-40b6-8862-8bd3871094d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:18:22.412100 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.auth.core [None req-2381e9b3-ea99-4c19-ab89-fdaa49ded91b None None] MFA Rules not processed for user `4174b35cc2ee4dd880b1e978de360413`. Rule list: `[]` (Enabled: `True`). {{(pid=80539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:18:22.417992 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-2381e9b3-ea99-4c19-ab89-fdaa49ded91b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:18:22.524535 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 892/1765] 10.222.0.23 () {58 vars in 957 bytes} [Thu May 19 12:18:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 295 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:18:22.531515 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19edae93-533d-40b6-8862-8bd3871094d9 None None] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:18:22.564509 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19edae93-533d-40b6-8862-8bd3871094d9 None None] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:18:22.595150 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa353f51-363f-4d08-a95f-d1178ad0c2c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:18:22.597180 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa353f51-363f-4d08-a95f-d1178ad0c2c8 admin admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:18:22.599540 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa353f51-363f-4d08-a95f-d1178ad0c2c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:18:22.600202 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa353f51-363f-4d08-a95f-d1178ad0c2c8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:18:22.603041 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa353f51-363f-4d08-a95f-d1178ad0c2c8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:18:22.605436 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa353f51-363f-4d08-a95f-d1178ad0c2c8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:18:22.609421 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa353f51-363f-4d08-a95f-d1178ad0c2c8 admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:18:22.639956 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 893/1766] 10.222.0.23 () {58 vars in 1152 bytes} [Thu May 19 12:18:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:18:22.646759 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa353f51-363f-4d08-a95f-d1178ad0c2c8 admin admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:18:22.690746 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa353f51-363f-4d08-a95f-d1178ad0c2c8 admin admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:18:22.746552 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d0ce44-396f-4895-984c-c23ef6a7db97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:18:22.749771 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d0ce44-396f-4895-984c-c23ef6a7db97 admin admin] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:18:22.752064 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d0ce44-396f-4895-984c-c23ef6a7db97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:18:22.754029 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d0ce44-396f-4895-984c-c23ef6a7db97 admin admin] PATH_INFO: `/v3/projects` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:18:22.760870 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d0ce44-396f-4895-984c-c23ef6a7db97 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:18:22.775693 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d0ce44-396f-4895-984c-c23ef6a7db97 admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:18:22.833127 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 874/1767] 10.222.0.23 () {58 vars in 957 bytes} [Thu May 19 12:18:22 2022] POST /identity/v3/auth/tokens => generated 3295 bytes in 548 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 12:18:22.870394 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2381e9b3-ea99-4c19-ab89-fdaa49ded91b None None] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:18:22.893557 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-2381e9b3-ea99-4c19-ab89-fdaa49ded91b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:18:23.106883 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:18:23.123457 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 894/1768] 10.222.0.23 () {60 vars in 1152 bytes} [Thu May 19 12:18:22 2022] POST /identity/v3/projects => generated 375 bytes in 475 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 12:18:23.125735 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d0ce44-396f-4895-984c-c23ef6a7db97 admin admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:18:23.147120 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2381e9b3-ea99-4c19-ab89-fdaa49ded91b None None] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:18:23.337332 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:18:23.413501 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9301f8-42f3-44f1-8456-03c20f2de4de service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8800254c0e784e42a791fb9abc274d79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd70d25bf69314820b8aaecb65576978a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:18:23.420832 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9301f8-42f3-44f1-8456-03c20f2de4de service cinder] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:18:23.422230 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9301f8-42f3-44f1-8456-03c20f2de4de service cinder] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:18:23.423928 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9301f8-42f3-44f1-8456-03c20f2de4de service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:18:23.430280 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d0ce44-396f-4895-984c-c23ef6a7db97 admin admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:18:23.455447 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d307edb-8528-44c4-a231-d56dae419ade admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:18:23.457223 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d307edb-8528-44c4-a231-d56dae419ade admin admin] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:18:23.457999 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d307edb-8528-44c4-a231-d56dae419ade admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:18:23.460341 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d307edb-8528-44c4-a231-d56dae419ade admin admin] PATH_INFO: `/v3/users` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:18:23.466497 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d307edb-8528-44c4-a231-d56dae419ade admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:18:23.471246 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d307edb-8528-44c4-a231-d56dae419ade admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:18:23.588773 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 895/1769] 10.222.0.23 () {60 vars in 1143 bytes} [Thu May 19 12:18:23 2022] POST /identity/v3/users => generated 288 bytes in 464 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 12:18:23.594298 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d307edb-8528-44c4-a231-d56dae419ade admin admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:18:23.658979 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d307edb-8528-44c4-a231-d56dae419ade admin admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:18:23.691461 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18446ec3-488f-4c7f-beac-ab873c8a3e2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:18:23.692637 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18446ec3-488f-4c7f-beac-ab873c8a3e2d admin admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:18:23.692884 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18446ec3-488f-4c7f-beac-ab873c8a3e2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:18:23.693174 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18446ec3-488f-4c7f-beac-ab873c8a3e2d admin admin] PATH_INFO: `/v3/roles` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:18:23.694041 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18446ec3-488f-4c7f-beac-ab873c8a3e2d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:18:23.696279 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18446ec3-488f-4c7f-beac-ab873c8a3e2d admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:18:23.720785 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 896/1770] 10.222.0.23 () {58 vars in 1121 bytes} [Thu May 19 12:18:23 2022] GET /identity/v3/roles => generated 1420 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 12:18:23.725326 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18446ec3-488f-4c7f-beac-ab873c8a3e2d admin admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:18:23.733292 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-18446ec3-488f-4c7f-beac-ab873c8a3e2d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:18:23.745333 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9301f8-42f3-44f1-8456-03c20f2de4de service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:18:23.746846 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9301f8-42f3-44f1-8456-03c20f2de4de service cinder] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:18:23.783955 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18446ec3-488f-4c7f-beac-ab873c8a3e2d admin admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:18:23.787107 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 875/1771] 10.222.0.23 () {60 vars in 1365 bytes} [Thu May 19 12:18:22 2022] GET /identity/v3/auth/tokens => generated 3295 bytes in 922 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:18:23.789138 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9301f8-42f3-44f1-8456-03c20f2de4de service cinder] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:18:23.799157 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d77f71-0734-46f5-a68d-6d7b54896a17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:18:23.800382 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d77f71-0734-46f5-a68d-6d7b54896a17 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:18:23.800814 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d77f71-0734-46f5-a68d-6d7b54896a17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:18:23.803429 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d77f71-0734-46f5-a68d-6d7b54896a17 admin admin] PATH_INFO: `/v3/projects/51937f43dc4c4f6b8d174fa3dbe47baa` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:18:23.806664 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d77f71-0734-46f5-a68d-6d7b54896a17 admin admin] RBAC: Authorizing `identity:delete_project(project_id=51937f43dc4c4f6b8d174fa3dbe47baa)` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:18:23.808699 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d77f71-0734-46f5-a68d-6d7b54896a17 admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:18:23.822723 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9301f8-42f3-44f1-8456-03c20f2de4de service cinder] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:18:23.836964 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe36543-253c-489d-bf41-5c7eb55c5ae8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:18:23.838087 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe36543-253c-489d-bf41-5c7eb55c5ae8 admin admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:18:23.840247 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe36543-253c-489d-bf41-5c7eb55c5ae8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:18:23.840746 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe36543-253c-489d-bf41-5c7eb55c5ae8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:18:23.841799 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe36543-253c-489d-bf41-5c7eb55c5ae8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:18:23.846190 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe36543-253c-489d-bf41-5c7eb55c5ae8 admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:18:23.875616 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 876/1772] 10.222.0.23 () {58 vars in 1121 bytes} [Thu May 19 12:18:23 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 12:18:23.880085 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe36543-253c-489d-bf41-5c7eb55c5ae8 admin admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:18:23.916201 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe36543-253c-489d-bf41-5c7eb55c5ae8 admin admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:18:23.926402 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f2f106-be6b-4166-8f18-7ec339fa3c30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:18:23.927515 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f2f106-be6b-4166-8f18-7ec339fa3c30 admin admin] REQUEST_METHOD: `PUT` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:18:23.928052 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f2f106-be6b-4166-8f18-7ec339fa3c30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:18:23.928493 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f2f106-be6b-4166-8f18-7ec339fa3c30 admin admin] PATH_INFO: `/v3/projects/4798f6c0018540c38a0d0c6276dccafa/users/7f98aebf55bc4f709b3b6e644077371c/roles/259eb400cdb4439db72d14456ae9cd08` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:18:23.988842 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f2f106-be6b-4166-8f18-7ec339fa3c30 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4798f6c0018540c38a0d0c6276dccafa, user_id=7f98aebf55bc4f709b3b6e644077371c, role_id=259eb400cdb4439db72d14456ae9cd08)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:18:23.992401 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f2f106-be6b-4166-8f18-7ec339fa3c30 admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:18:24.123338 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 877/1773] 10.222.0.23 () {60 vars in 1482 bytes} [Thu May 19 12:18:23 2022] PUT /identity/v3/projects/4798f6c0018540c38a0d0c6276dccafa/users/7f98aebf55bc4f709b3b6e644077371c/roles/259eb400cdb4439db72d14456ae9cd08 => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:18:24.124818 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f2f106-be6b-4166-8f18-7ec339fa3c30 admin admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:18:24.163751 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-a7d77f71-0734-46f5-a68d-6d7b54896a17 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '51937f43dc4c4f6b8d174fa3dbe47baa', 'request_id': 'req-a7d77f71-0734-46f5-a68d-6d7b54896a17'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:18:24.180212 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-a7d77f71-0734-46f5-a68d-6d7b54896a17 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '51937f43dc4c4f6b8d174fa3dbe47baa', 'request_id': 'req-a7d77f71-0734-46f5-a68d-6d7b54896a17'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:18:24.206072 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-a7d77f71-0734-46f5-a68d-6d7b54896a17 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '51937f43dc4c4f6b8d174fa3dbe47baa', 'request_id': 'req-a7d77f71-0734-46f5-a68d-6d7b54896a17'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:18:24.250181 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f2f106-be6b-4166-8f18-7ec339fa3c30 admin admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:18:24.363158 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d2393b-9687-47dd-a073-f651215ae5c8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8800254c0e784e42a791fb9abc274d79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd70d25bf69314820b8aaecb65576978a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:18:24.364851 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d2393b-9687-47dd-a073-f651215ae5c8 service cinder] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:18:24.365524 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d2393b-9687-47dd-a073-f651215ae5c8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:18:24.366194 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d2393b-9687-47dd-a073-f651215ae5c8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:18:24.372119 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-57d2393b-9687-47dd-a073-f651215ae5c8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:18:24.406435 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-a7d77f71-0734-46f5-a68d-6d7b54896a17 admin admin] The token cache is being invalidate because project 51937f43dc4c4f6b8d174fa3dbe47baa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80540) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 12:18:24.406738 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-a7d77f71-0734-46f5-a68d-6d7b54896a17 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:18:24.408387 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-a7d77f71-0734-46f5-a68d-6d7b54896a17 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:18:24.419064 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 897/1774] 10.222.0.23 () {58 vars in 1232 bytes} [Thu May 19 12:18:23 2022] DELETE /identity/v3/projects/51937f43dc4c4f6b8d174fa3dbe47baa => generated 0 bytes in 690 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:18:24.419064 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5297b4-09f3-4dce-8d90-5d4c3045a4c2 None None] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:18:24.419064 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5297b4-09f3-4dce-8d90-5d4c3045a4c2 None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:18:24.419064 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5297b4-09f3-4dce-8d90-5d4c3045a4c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:18:24.483335 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.auth.core [None req-2e5297b4-09f3-4dce-8d90-5d4c3045a4c2 None None] MFA Rules not processed for user `7f98aebf55bc4f709b3b6e644077371c`. Rule list: `[]` (Enabled: `True`). {{(pid=80540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:18:24.485659 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-2e5297b4-09f3-4dce-8d90-5d4c3045a4c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:18:24.531519 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:18:24.643359 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:18:24.759512 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 898/1775] 10.222.0.23 () {58 vars in 957 bytes} [Thu May 19 12:18:24 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 343 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:18:24.762268 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5297b4-09f3-4dce-8d90-5d4c3045a4c2 None None] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:18:24.766486 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-2e5297b4-09f3-4dce-8d90-5d4c3045a4c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:18:24.770768 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d2393b-9687-47dd-a073-f651215ae5c8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:18:24.776707 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d2393b-9687-47dd-a073-f651215ae5c8 service cinder] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:18:24.811374 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 878/1776] 10.222.0.23 () {60 vars in 1365 bytes} [Thu May 19 12:18:24 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 687 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:18:24.816152 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d2393b-9687-47dd-a073-f651215ae5c8 service cinder] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:18:24.824191 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-57d2393b-9687-47dd-a073-f651215ae5c8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:18:25.157276 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d2393b-9687-47dd-a073-f651215ae5c8 service cinder] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:18:25.166105 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77972503-d757-4f24-8eda-29e3c6098aa5 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '21697802f2be42f9baa8e07490d5ebdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd70d25bf69314820b8aaecb65576978a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:18:25.166105 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77972503-d757-4f24-8eda-29e3c6098aa5 service glance] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:18:25.166105 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77972503-d757-4f24-8eda-29e3c6098aa5 service glance] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:18:25.166105 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77972503-d757-4f24-8eda-29e3c6098aa5 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:18:25.175000 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5297b4-09f3-4dce-8d90-5d4c3045a4c2 None None] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:18:25.184499 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77972503-d757-4f24-8eda-29e3c6098aa5 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:18:25.207577 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77972503-d757-4f24-8eda-29e3c6098aa5 service glance] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:18:25.208151 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3aef01a-33cf-4e7e-bf9b-7b61985d641e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:18:25.208151 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3aef01a-33cf-4e7e-bf9b-7b61985d641e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:18:25.208151 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3aef01a-33cf-4e7e-bf9b-7b61985d641e admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:18:25.208151 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3aef01a-33cf-4e7e-bf9b-7b61985d641e admin admin] PATH_INFO: `/v3/users/c65fdbdb4a8d41b0aca274301c8fff1e` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:18:25.208733 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3aef01a-33cf-4e7e-bf9b-7b61985d641e admin admin] RBAC: Authorizing `identity:delete_user(user_id=c65fdbdb4a8d41b0aca274301c8fff1e)` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:18:25.208733 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3aef01a-33cf-4e7e-bf9b-7b61985d641e admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:18:25.208996 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 879/1777] 10.222.0.23 () {60 vars in 1365 bytes} [Thu May 19 12:18:24 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:18:25.970268 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-d3aef01a-33cf-4e7e-bf9b-7b61985d641e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c65fdbdb4a8d41b0aca274301c8fff1e'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:18:26.002116 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-d3aef01a-33cf-4e7e-bf9b-7b61985d641e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c65fdbdb4a8d41b0aca274301c8fff1e'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:18:26.003806 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-d3aef01a-33cf-4e7e-bf9b-7b61985d641e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c65fdbdb4a8d41b0aca274301c8fff1e'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:18:26.128110 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-d3aef01a-33cf-4e7e-bf9b-7b61985d641e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c65fdbdb4a8d41b0aca274301c8fff1e'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:18:26.158791 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-d3aef01a-33cf-4e7e-bf9b-7b61985d641e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c65fdbdb4a8d41b0aca274301c8fff1e'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:18:26.176162 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 899/1778] 10.222.0.23 () {58 vars in 1223 bytes} [Thu May 19 12:18:24 2022] DELETE /identity/v3/users/c65fdbdb4a8d41b0aca274301c8fff1e => generated 0 bytes in 1406 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:18:26.936474 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be0c106-f86f-4de6-aff8-830d7498d314 None None] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:18:26.937411 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be0c106-f86f-4de6-aff8-830d7498d314 None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:18:26.937724 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be0c106-f86f-4de6-aff8-830d7498d314 None None] PATH_INFO: `/` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:18:26.940858 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 880/1779] 10.222.0.23 () {54 vars in 921 bytes} [Thu May 19 12:18:26 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 12:18:26.954185 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c3a733-4101-47dc-9ca9-a8b436617a07 None None] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:18:26.954185 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c3a733-4101-47dc-9ca9-a8b436617a07 None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:18:26.954185 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c3a733-4101-47dc-9ca9-a8b436617a07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:18:26.999882 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.auth.core [None req-69c3a733-4101-47dc-9ca9-a8b436617a07 None None] MFA Rules not processed for user `34a98d75d84b4c67981377e0f26eca62`. Rule list: `[]` (Enabled: `True`). {{(pid=80540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:18:27.001143 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-69c3a733-4101-47dc-9ca9-a8b436617a07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:18:27.262737 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:18:27.308047 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 900/1780] 10.222.0.23 () {58 vars in 1018 bytes} [Thu May 19 12:18:26 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 356 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 12:18:27.511042 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb1304f-2f37-4d67-a25b-feadb63a6678 None None] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:18:27.511932 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb1304f-2f37-4d67-a25b-feadb63a6678 None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:18:27.512703 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb1304f-2f37-4d67-a25b-feadb63a6678 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:18:27.550178 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.auth.core [None req-9bb1304f-2f37-4d67-a25b-feadb63a6678 None None] MFA Rules not processed for user `dd39ddcf4ab045b98a737ee529a2dcce`. Rule list: `[]` (Enabled: `True`). {{(pid=80539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:18:27.552962 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-9bb1304f-2f37-4d67-a25b-feadb63a6678 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:18:27.593348 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c3a733-4101-47dc-9ca9-a8b436617a07 None None] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:18:27.596438 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-69c3a733-4101-47dc-9ca9-a8b436617a07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:18:27.733037 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 881/1781] 10.222.0.23 () {58 vars in 957 bytes} [Thu May 19 12:18:27 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:18:27.778919 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c3a733-4101-47dc-9ca9-a8b436617a07 None None] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:18:27.791494 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb505c5-df2e-4773-b8d3-e4ca95558077 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:18:27.792924 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb505c5-df2e-4773-b8d3-e4ca95558077 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:18:27.793379 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb505c5-df2e-4773-b8d3-e4ca95558077 admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:18:27.793961 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb505c5-df2e-4773-b8d3-e4ca95558077 admin admin] PATH_INFO: `/v3/projects/86d65ab5f7aa4fec9fcabf9264e4f819` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:18:27.795654 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb505c5-df2e-4773-b8d3-e4ca95558077 admin admin] RBAC: Authorizing `identity:delete_project(project_id=86d65ab5f7aa4fec9fcabf9264e4f819)` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:18:27.808354 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb505c5-df2e-4773-b8d3-e4ca95558077 admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:18:27.816285 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb1304f-2f37-4d67-a25b-feadb63a6678 None None] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:18:27.825327 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-9bb1304f-2f37-4d67-a25b-feadb63a6678 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:18:27.873750 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb1304f-2f37-4d67-a25b-feadb63a6678 None None] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:18:27.881263 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65d98b1-fc41-4c7f-913c-6671d721699e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:18:27.882317 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65d98b1-fc41-4c7f-913c-6671d721699e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:18:27.883050 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65d98b1-fc41-4c7f-913c-6671d721699e admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:18:27.883576 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65d98b1-fc41-4c7f-913c-6671d721699e admin admin] PATH_INFO: `/v3/users/23cf1bed54db44c29d91fb87d40c9f25` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:18:27.894317 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65d98b1-fc41-4c7f-913c-6671d721699e admin admin] RBAC: Authorizing `identity:delete_user(user_id=23cf1bed54db44c29d91fb87d40c9f25)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:18:27.894317 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65d98b1-fc41-4c7f-913c-6671d721699e admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:18:28.002624 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-3eb505c5-df2e-4773-b8d3-e4ca95558077 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '86d65ab5f7aa4fec9fcabf9264e4f819', 'request_id': 'req-3eb505c5-df2e-4773-b8d3-e4ca95558077'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:18:28.010121 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-3eb505c5-df2e-4773-b8d3-e4ca95558077 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '86d65ab5f7aa4fec9fcabf9264e4f819', 'request_id': 'req-3eb505c5-df2e-4773-b8d3-e4ca95558077'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:18:28.016989 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-3eb505c5-df2e-4773-b8d3-e4ca95558077 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '86d65ab5f7aa4fec9fcabf9264e4f819', 'request_id': 'req-3eb505c5-df2e-4773-b8d3-e4ca95558077'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:18:28.150402 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-3eb505c5-df2e-4773-b8d3-e4ca95558077 admin admin] The token cache is being invalidate because project 86d65ab5f7aa4fec9fcabf9264e4f819 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80540) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 12:18:28.151597 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-3eb505c5-df2e-4773-b8d3-e4ca95558077 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:18:28.154024 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-3eb505c5-df2e-4773-b8d3-e4ca95558077 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:18:28.159626 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 901/1782] 10.222.0.23 () {58 vars in 1232 bytes} [Thu May 19 12:18:27 2022] DELETE /identity/v3/projects/86d65ab5f7aa4fec9fcabf9264e4f819 => generated 0 bytes in 567 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:18:28.161174 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb505c5-df2e-4773-b8d3-e4ca95558077 admin admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:18:28.165987 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-3eb505c5-df2e-4773-b8d3-e4ca95558077 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:18:28.251820 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-e65d98b1-fc41-4c7f-913c-6671d721699e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '23cf1bed54db44c29d91fb87d40c9f25'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:18:28.289162 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-e65d98b1-fc41-4c7f-913c-6671d721699e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '23cf1bed54db44c29d91fb87d40c9f25'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:18:28.295625 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-e65d98b1-fc41-4c7f-913c-6671d721699e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '23cf1bed54db44c29d91fb87d40c9f25'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:18:28.375745 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb505c5-df2e-4773-b8d3-e4ca95558077 admin admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:18:28.391259 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a284303-8e56-4d06-801b-5e2b7ee029a9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34a98d75d84b4c67981377e0f26eca62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd70d25bf69314820b8aaecb65576978a', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:18:28.392482 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a284303-8e56-4d06-801b-5e2b7ee029a9 service nova] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:18:28.393360 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a284303-8e56-4d06-801b-5e2b7ee029a9 service nova] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:18:28.393930 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a284303-8e56-4d06-801b-5e2b7ee029a9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:18:28.394335 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-e65d98b1-fc41-4c7f-913c-6671d721699e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '23cf1bed54db44c29d91fb87d40c9f25'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:18:28.397150 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:18:28.397650 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-4a284303-8e56-4d06-801b-5e2b7ee029a9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:18:28.409095 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-e65d98b1-fc41-4c7f-913c-6671d721699e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '23cf1bed54db44c29d91fb87d40c9f25'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:18:28.427571 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 882/1783] 10.222.0.23 () {58 vars in 1223 bytes} [Thu May 19 12:18:27 2022] DELETE /identity/v3/users/23cf1bed54db44c29d91fb87d40c9f25 => generated 0 bytes in 612 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 12:18:28.433013 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a284303-8e56-4d06-801b-5e2b7ee029a9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:18:28.434866 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a284303-8e56-4d06-801b-5e2b7ee029a9 service nova] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:18:28.446860 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d2ecaf-212b-4cef-8fd9-2b3f82e3a503 None None] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:18:28.446860 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d2ecaf-212b-4cef-8fd9-2b3f82e3a503 None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:18:28.446860 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d2ecaf-212b-4cef-8fd9-2b3f82e3a503 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:18:28.452961 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 902/1784] 10.222.0.23 () {60 vars in 1365 bytes} [Thu May 19 12:18:28 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:18:28.501051 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.auth.core [None req-04d2ecaf-212b-4cef-8fd9-2b3f82e3a503 None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:18:28.505371 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-04d2ecaf-212b-4cef-8fd9-2b3f82e3a503 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:18:28.626899 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 883/1785] 10.222.0.23 () {58 vars in 957 bytes} [Thu May 19 12:18:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:18:28.637078 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24143076-b5f1-4ef9-acc2-37a258b6c652 None None] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:18:28.643096 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24143076-b5f1-4ef9-acc2-37a258b6c652 None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:18:28.643096 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24143076-b5f1-4ef9-acc2-37a258b6c652 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:18:28.662074 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.auth.core [None req-24143076-b5f1-4ef9-acc2-37a258b6c652 None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:18:28.663499 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-24143076-b5f1-4ef9-acc2-37a258b6c652 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:18:28.676299 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 903/1786] 10.222.0.23 () {58 vars in 957 bytes} [Thu May 19 12:18:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 12:18:28.681345 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d2ecaf-212b-4cef-8fd9-2b3f82e3a503 None None] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:18:28.696362 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d2ecaf-212b-4cef-8fd9-2b3f82e3a503 None None] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:18:28.703740 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac831ad-4f69-454f-b439-2a2af5c6e177 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:18:28.704640 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac831ad-4f69-454f-b439-2a2af5c6e177 admin admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:18:28.705026 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac831ad-4f69-454f-b439-2a2af5c6e177 admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:18:28.706026 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac831ad-4f69-454f-b439-2a2af5c6e177 admin admin] PATH_INFO: `/v3/domains` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:18:28.706888 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac831ad-4f69-454f-b439-2a2af5c6e177 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:18:28.707310 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac831ad-4f69-454f-b439-2a2af5c6e177 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:18:28.710030 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac831ad-4f69-454f-b439-2a2af5c6e177 admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:18:28.789821 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 884/1787] 10.222.0.23 () {58 vars in 1152 bytes} [Thu May 19 12:18:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:18:28.791625 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24143076-b5f1-4ef9-acc2-37a258b6c652 None None] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:18:28.802641 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24143076-b5f1-4ef9-acc2-37a258b6c652 None None] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:18:28.809181 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0a4f9d-b2d6-436b-adac-8969aaa9d5da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:18:28.810287 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0a4f9d-b2d6-436b-adac-8969aaa9d5da admin admin] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:18:28.810642 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0a4f9d-b2d6-436b-adac-8969aaa9d5da admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:18:28.811426 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0a4f9d-b2d6-436b-adac-8969aaa9d5da admin admin] PATH_INFO: `/v3/projects` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:18:28.813464 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0a4f9d-b2d6-436b-adac-8969aaa9d5da admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:18:28.816965 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0a4f9d-b2d6-436b-adac-8969aaa9d5da admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:18:28.823227 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:18:28.844007 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac831ad-4f69-454f-b439-2a2af5c6e177 admin admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:18:28.846103 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-1ac831ad-4f69-454f-b439-2a2af5c6e177 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:18:28.890548 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac831ad-4f69-454f-b439-2a2af5c6e177 admin admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:18:28.896898 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9da33d-fa2d-4671-b235-428f39528393 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:18:28.897877 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9da33d-fa2d-4671-b235-428f39528393 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:18:28.898178 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9da33d-fa2d-4671-b235-428f39528393 admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:18:28.898479 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9da33d-fa2d-4671-b235-428f39528393 admin admin] PATH_INFO: `/v3/projects/f8ff69832f884971bfaa8fcbabb6f059` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:18:28.903006 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9da33d-fa2d-4671-b235-428f39528393 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f8ff69832f884971bfaa8fcbabb6f059)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:18:28.909326 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9da33d-fa2d-4671-b235-428f39528393 admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:18:28.970111 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 904/1788] 10.222.0.23 () {60 vars in 1152 bytes} [Thu May 19 12:18:28 2022] POST /identity/v3/projects => generated 387 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 12:18:28.970111 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f261dda-610b-4633-bb66-49b72e8e22bb None None] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:18:28.970111 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f261dda-610b-4633-bb66-49b72e8e22bb None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:18:28.971712 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f261dda-610b-4633-bb66-49b72e8e22bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:18:29.017173 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.auth.core [None req-2f261dda-610b-4633-bb66-49b72e8e22bb None None] MFA Rules not processed for user `9d762cce00ae4e468c358b0437f35deb`. Rule list: `[]` (Enabled: `True`). {{(pid=80540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:18:29.019376 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-2f261dda-610b-4633-bb66-49b72e8e22bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:18:29.100206 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-fa9da33d-fa2d-4671-b235-428f39528393 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f8ff69832f884971bfaa8fcbabb6f059', 'request_id': 'req-fa9da33d-fa2d-4671-b235-428f39528393'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:18:29.106563 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-fa9da33d-fa2d-4671-b235-428f39528393 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f8ff69832f884971bfaa8fcbabb6f059', 'request_id': 'req-fa9da33d-fa2d-4671-b235-428f39528393'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:18:29.127941 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 905/1789] 10.222.0.23 () {58 vars in 957 bytes} [Thu May 19 12:18:28 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 12:18:29.127941 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f261dda-610b-4633-bb66-49b72e8e22bb None None] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:18:29.180396 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-fa9da33d-fa2d-4671-b235-428f39528393 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f8ff69832f884971bfaa8fcbabb6f059', 'request_id': 'req-fa9da33d-fa2d-4671-b235-428f39528393'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:18:29.203456 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-fa9da33d-fa2d-4671-b235-428f39528393 admin admin] The token cache is being invalidate because project f8ff69832f884971bfaa8fcbabb6f059 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80539) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 12:18:29.204586 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-fa9da33d-fa2d-4671-b235-428f39528393 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:18:29.205717 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-fa9da33d-fa2d-4671-b235-428f39528393 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:18:29.208885 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 885/1790] 10.222.0.23 () {58 vars in 1232 bytes} [Thu May 19 12:18:28 2022] DELETE /identity/v3/projects/f8ff69832f884971bfaa8fcbabb6f059 => generated 0 bytes in 367 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:18:29.211705 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49aba8c6-eb28-4f3f-9394-c0ec44dfc42b None None] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:18:29.214735 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49aba8c6-eb28-4f3f-9394-c0ec44dfc42b None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:18:29.214735 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49aba8c6-eb28-4f3f-9394-c0ec44dfc42b None None] PATH_INFO: `/` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:18:29.221904 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 886/1791] 10.222.0.23 () {54 vars in 912 bytes} [Thu May 19 12:18:29 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 12:18:29.223681 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49aba8c6-eb28-4f3f-9394-c0ec44dfc42b None None] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:18:29.227325 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-49aba8c6-eb28-4f3f-9394-c0ec44dfc42b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:18:29.254965 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f261dda-610b-4633-bb66-49b72e8e22bb None None] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:18:29.367765 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49aba8c6-eb28-4f3f-9394-c0ec44dfc42b None None] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:18:29.378718 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f06214-235f-459d-8349-51f9f5f246e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:18:29.380419 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f06214-235f-459d-8349-51f9f5f246e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:18:29.382492 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f06214-235f-459d-8349-51f9f5f246e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:18:29.382492 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f06214-235f-459d-8349-51f9f5f246e0 admin admin] PATH_INFO: `/v3/users/8c99000c8e6f4a98b430bfd479f87287` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:18:29.387277 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f06214-235f-459d-8349-51f9f5f246e0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8c99000c8e6f4a98b430bfd479f87287)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:18:29.392275 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f06214-235f-459d-8349-51f9f5f246e0 admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:18:29.396208 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:18:29.402446 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1069f452-b2a9-442c-b5c6-df3459443ae8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:18:29.403517 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1069f452-b2a9-442c-b5c6-df3459443ae8 admin admin] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:18:29.404014 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1069f452-b2a9-442c-b5c6-df3459443ae8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:18:29.404952 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1069f452-b2a9-442c-b5c6-df3459443ae8 admin admin] PATH_INFO: `/v3/users` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:18:29.407380 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1069f452-b2a9-442c-b5c6-df3459443ae8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:18:29.411778 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1069f452-b2a9-442c-b5c6-df3459443ae8 admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:18:29.533525 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 906/1792] 10.222.0.23 () {60 vars in 1143 bytes} [Thu May 19 12:18:29 2022] POST /identity/v3/users => generated 294 bytes in 408 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 12:18:29.535190 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d5f1de-d15f-4b8b-9930-a16d20b9ffca None None] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:18:29.535587 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d5f1de-d15f-4b8b-9930-a16d20b9ffca None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:18:29.536127 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d5f1de-d15f-4b8b-9930-a16d20b9ffca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:18:29.559117 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.auth.core [None req-c6d5f1de-d15f-4b8b-9930-a16d20b9ffca None None] MFA Rules not processed for user `1a00b3fa7497456895e5e0bf105c4fc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:18:29.560457 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-c6d5f1de-d15f-4b8b-9930-a16d20b9ffca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:18:29.595349 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:18:29.640777 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 907/1793] 10.222.0.23 () {58 vars in 1009 bytes} [Thu May 19 12:18:29 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:18:29.641808 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d5f1de-d15f-4b8b-9930-a16d20b9ffca None None] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:18:29.644069 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-c6d5f1de-d15f-4b8b-9930-a16d20b9ffca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:18:29.672415 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d5f1de-d15f-4b8b-9930-a16d20b9ffca None None] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:18:29.676750 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-43f06214-235f-459d-8349-51f9f5f246e0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8c99000c8e6f4a98b430bfd479f87287'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:18:29.682631 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12f5430-0195-4752-a36b-dc80b8d5cb27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:18:29.683478 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12f5430-0195-4752-a36b-dc80b8d5cb27 admin admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:18:29.684059 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12f5430-0195-4752-a36b-dc80b8d5cb27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:18:29.684552 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12f5430-0195-4752-a36b-dc80b8d5cb27 admin admin] PATH_INFO: `/v3/roles` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:18:29.689261 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12f5430-0195-4752-a36b-dc80b8d5cb27 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:18:29.689261 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12f5430-0195-4752-a36b-dc80b8d5cb27 admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:18:29.702461 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-43f06214-235f-459d-8349-51f9f5f246e0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8c99000c8e6f4a98b430bfd479f87287'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:18:29.703646 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-43f06214-235f-459d-8349-51f9f5f246e0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8c99000c8e6f4a98b430bfd479f87287'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:18:29.706118 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 908/1794] 10.222.0.23 () {58 vars in 1121 bytes} [Thu May 19 12:18:29 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 12:18:29.707524 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12f5430-0195-4752-a36b-dc80b8d5cb27 admin admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:18:29.712207 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-c12f5430-0195-4752-a36b-dc80b8d5cb27 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:18:29.734368 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12f5430-0195-4752-a36b-dc80b8d5cb27 admin admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:18:29.738707 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ae9ece-a41f-4a90-b9bc-26436e625ae1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a00b3fa7497456895e5e0bf105c4fc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd70d25bf69314820b8aaecb65576978a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:18:29.739510 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ae9ece-a41f-4a90-b9bc-26436e625ae1 service neutron] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:18:29.739964 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ae9ece-a41f-4a90-b9bc-26436e625ae1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:18:29.740312 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ae9ece-a41f-4a90-b9bc-26436e625ae1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:18:29.750729 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ae9ece-a41f-4a90-b9bc-26436e625ae1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:18:29.752787 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ae9ece-a41f-4a90-b9bc-26436e625ae1 service neutron] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:18:29.765158 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 909/1795] 10.222.0.23 () {60 vars in 1365 bytes} [Thu May 19 12:18:29 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:18:29.766499 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ae9ece-a41f-4a90-b9bc-26436e625ae1 service neutron] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:18:29.788133 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ae9ece-a41f-4a90-b9bc-26436e625ae1 service neutron] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:18:29.796833 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b240588-2711-4d17-9520-27001dabfcb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:18:29.797921 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b240588-2711-4d17-9520-27001dabfcb5 admin admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:18:29.798402 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b240588-2711-4d17-9520-27001dabfcb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:18:29.798836 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b240588-2711-4d17-9520-27001dabfcb5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:18:29.800555 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b240588-2711-4d17-9520-27001dabfcb5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:18:29.802230 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b240588-2711-4d17-9520-27001dabfcb5 admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:18:29.812462 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-43f06214-235f-459d-8349-51f9f5f246e0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8c99000c8e6f4a98b430bfd479f87287'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:18:29.832685 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 910/1796] 10.222.0.23 () {58 vars in 1121 bytes} [Thu May 19 12:18:29 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 12:18:29.837993 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b240588-2711-4d17-9520-27001dabfcb5 admin admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:18:29.850634 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-43f06214-235f-459d-8349-51f9f5f246e0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8c99000c8e6f4a98b430bfd479f87287'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:18:29.861599 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b240588-2711-4d17-9520-27001dabfcb5 admin admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:18:29.865796 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 887/1797] 10.222.0.23 () {58 vars in 1223 bytes} [Thu May 19 12:18:29 2022] DELETE /identity/v3/users/8c99000c8e6f4a98b430bfd479f87287 => generated 0 bytes in 643 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:18:29.996814 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21e2ff9-a739-45be-8021-3da193b6dba3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:18:29.998054 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21e2ff9-a739-45be-8021-3da193b6dba3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:18:29.998054 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21e2ff9-a739-45be-8021-3da193b6dba3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:18:29.998350 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21e2ff9-a739-45be-8021-3da193b6dba3 admin admin] PATH_INFO: `/v3/projects/1c781d06772d4a11b11d7bacab2cdcb8/users/57ae03648f2b45588ff418f7daef3120/roles/259eb400cdb4439db72d14456ae9cd08` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:18:30.066378 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21e2ff9-a739-45be-8021-3da193b6dba3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1c781d06772d4a11b11d7bacab2cdcb8, user_id=57ae03648f2b45588ff418f7daef3120, role_id=259eb400cdb4439db72d14456ae9cd08)` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:18:30.076040 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21e2ff9-a739-45be-8021-3da193b6dba3 admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:18:30.190398 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 911/1798] 10.222.0.23 () {60 vars in 1482 bytes} [Thu May 19 12:18:29 2022] PUT /identity/v3/projects/1c781d06772d4a11b11d7bacab2cdcb8/users/57ae03648f2b45588ff418f7daef3120/roles/259eb400cdb4439db72d14456ae9cd08 => generated 0 bytes in 353 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:18:30.215399 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befa7086-accc-42a1-bb46-d998164da147 None None] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:18:30.218545 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befa7086-accc-42a1-bb46-d998164da147 None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:18:30.218545 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befa7086-accc-42a1-bb46-d998164da147 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:18:30.262839 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.auth.core [None req-befa7086-accc-42a1-bb46-d998164da147 None None] MFA Rules not processed for user `57ae03648f2b45588ff418f7daef3120`. Rule list: `[]` (Enabled: `True`). {{(pid=80539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:18:30.262839 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-befa7086-accc-42a1-bb46-d998164da147 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:18:30.263866 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21e2ff9-a739-45be-8021-3da193b6dba3 admin admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:18:30.265705 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-b21e2ff9-a739-45be-8021-3da193b6dba3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:18:30.436928 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21e2ff9-a739-45be-8021-3da193b6dba3 admin admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:18:30.446578 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67cd1b7-5b90-4012-b5a6-cc2c3c5b4285 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:18:30.447619 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67cd1b7-5b90-4012-b5a6-cc2c3c5b4285 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:18:30.448031 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67cd1b7-5b90-4012-b5a6-cc2c3c5b4285 admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:18:30.448384 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67cd1b7-5b90-4012-b5a6-cc2c3c5b4285 admin admin] PATH_INFO: `/v3/users/e6367f23eb0c4c1e88fd349c4d3fc073` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:18:30.452650 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e67cd1b7-5b90-4012-b5a6-cc2c3c5b4285 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e6367f23eb0c4c1e88fd349c4d3fc073)` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:18:30.455574 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e67cd1b7-5b90-4012-b5a6-cc2c3c5b4285 admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:18:30.468280 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 888/1799] 10.222.0.23 () {58 vars in 957 bytes} [Thu May 19 12:18:30 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 268 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:18:30.470839 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befa7086-accc-42a1-bb46-d998164da147 None None] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:18:30.473687 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-befa7086-accc-42a1-bb46-d998164da147 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:18:30.507858 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befa7086-accc-42a1-bb46-d998164da147 None None] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:18:30.510174 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc9fefc-e0ae-4ac7-8482-4562f3ca25cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:18:30.511584 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc9fefc-e0ae-4ac7-8482-4562f3ca25cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:18:30.512091 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc9fefc-e0ae-4ac7-8482-4562f3ca25cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:18:30.512644 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc9fefc-e0ae-4ac7-8482-4562f3ca25cb admin admin] PATH_INFO: `/v3/projects/96fcdc6c8ade449c828f08da5aba92e6` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:18:30.514626 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc9fefc-e0ae-4ac7-8482-4562f3ca25cb admin admin] RBAC: Authorizing `identity:delete_project(project_id=96fcdc6c8ade449c828f08da5aba92e6)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:18:30.517957 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc9fefc-e0ae-4ac7-8482-4562f3ca25cb admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:18:31.746304 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=94,keystone:INSERT=1,keystone:DELETE=5 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:18:31.785422 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-5bc9fefc-e0ae-4ac7-8482-4562f3ca25cb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '96fcdc6c8ade449c828f08da5aba92e6', 'request_id': 'req-5bc9fefc-e0ae-4ac7-8482-4562f3ca25cb'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:18:31.813188 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-5bc9fefc-e0ae-4ac7-8482-4562f3ca25cb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '96fcdc6c8ade449c828f08da5aba92e6', 'request_id': 'req-5bc9fefc-e0ae-4ac7-8482-4562f3ca25cb'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:18:31.984534 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-5bc9fefc-e0ae-4ac7-8482-4562f3ca25cb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '96fcdc6c8ade449c828f08da5aba92e6', 'request_id': 'req-5bc9fefc-e0ae-4ac7-8482-4562f3ca25cb'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:18:31.987516 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:18:32.003857 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-e67cd1b7-5b90-4012-b5a6-cc2c3c5b4285 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e6367f23eb0c4c1e88fd349c4d3fc073'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:18:32.006661 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-5bc9fefc-e0ae-4ac7-8482-4562f3ca25cb admin admin] The token cache is being invalidate because project 96fcdc6c8ade449c828f08da5aba92e6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80539) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 12:18:32.008044 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-5bc9fefc-e0ae-4ac7-8482-4562f3ca25cb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:18:32.010585 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-5bc9fefc-e0ae-4ac7-8482-4562f3ca25cb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:18:32.013690 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 889/1800] 10.222.0.23 () {58 vars in 1232 bytes} [Thu May 19 12:18:30 2022] DELETE /identity/v3/projects/96fcdc6c8ade449c828f08da5aba92e6 => generated 0 bytes in 1544 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:18:32.015330 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc9fefc-e0ae-4ac7-8482-4562f3ca25cb admin admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:18:32.017599 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-5bc9fefc-e0ae-4ac7-8482-4562f3ca25cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:18:32.033720 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-e67cd1b7-5b90-4012-b5a6-cc2c3c5b4285 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e6367f23eb0c4c1e88fd349c4d3fc073'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:18:32.034568 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-e67cd1b7-5b90-4012-b5a6-cc2c3c5b4285 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e6367f23eb0c4c1e88fd349c4d3fc073'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:18:32.130146 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-e67cd1b7-5b90-4012-b5a6-cc2c3c5b4285 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e6367f23eb0c4c1e88fd349c4d3fc073'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:18:32.137619 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc9fefc-e0ae-4ac7-8482-4562f3ca25cb admin admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:18:32.142345 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce56d9a-1f04-4fc6-b722-9ede450303ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:18:32.143051 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce56d9a-1f04-4fc6-b722-9ede450303ca admin admin] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:18:32.143335 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce56d9a-1f04-4fc6-b722-9ede450303ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:18:32.143616 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce56d9a-1f04-4fc6-b722-9ede450303ca admin admin] PATH_INFO: `/v3/projects` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:18:32.144405 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce56d9a-1f04-4fc6-b722-9ede450303ca admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:18:32.146446 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce56d9a-1f04-4fc6-b722-9ede450303ca admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:18:32.152733 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-e67cd1b7-5b90-4012-b5a6-cc2c3c5b4285 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e6367f23eb0c4c1e88fd349c4d3fc073'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:18:32.174319 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 912/1801] 10.222.0.23 () {58 vars in 1223 bytes} [Thu May 19 12:18:30 2022] DELETE /identity/v3/users/e6367f23eb0c4c1e88fd349c4d3fc073 => generated 0 bytes in 1907 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:18:32.174319 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8adfd12-d1ff-435e-803a-dbad2e9de444 None None] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:18:32.174319 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8adfd12-d1ff-435e-803a-dbad2e9de444 None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:18:32.174319 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8adfd12-d1ff-435e-803a-dbad2e9de444 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:18:32.207899 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.auth.core [None req-a8adfd12-d1ff-435e-803a-dbad2e9de444 None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:18:32.209466 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-a8adfd12-d1ff-435e-803a-dbad2e9de444 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:18:32.291242 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 890/1802] 10.222.0.23 () {60 vars in 1152 bytes} [Thu May 19 12:18:32 2022] POST /identity/v3/projects => generated 383 bytes in 275 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 12:18:32.293429 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e894b15b-52f3-4568-b1a4-33501f204bdf None None] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:18:32.294049 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e894b15b-52f3-4568-b1a4-33501f204bdf None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:18:32.294300 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e894b15b-52f3-4568-b1a4-33501f204bdf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:18:32.334345 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.auth.core [None req-e894b15b-52f3-4568-b1a4-33501f204bdf None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:18:32.336726 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-e894b15b-52f3-4568-b1a4-33501f204bdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:18:32.383858 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 913/1803] 10.222.0.23 () {58 vars in 957 bytes} [Thu May 19 12:18:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:18:32.385102 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8adfd12-d1ff-435e-803a-dbad2e9de444 None None] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:18:32.387566 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-a8adfd12-d1ff-435e-803a-dbad2e9de444 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:18:32.417318 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8adfd12-d1ff-435e-803a-dbad2e9de444 None None] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:18:32.455336 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 891/1804] 10.222.0.23 () {58 vars in 957 bytes} [Thu May 19 12:18:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:18:32.455336 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24fdfb2-58ff-4f54-ae62-0c3f3333e6c6 None None] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:18:32.455336 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24fdfb2-58ff-4f54-ae62-0c3f3333e6c6 None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:18:32.455336 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24fdfb2-58ff-4f54-ae62-0c3f3333e6c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:18:32.480819 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.auth.core [None req-d24fdfb2-58ff-4f54-ae62-0c3f3333e6c6 None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:18:32.482249 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-d24fdfb2-58ff-4f54-ae62-0c3f3333e6c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:18:32.490785 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 892/1805] 10.222.0.23 () {58 vars in 957 bytes} [Thu May 19 12:18:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:18:32.492386 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24fdfb2-58ff-4f54-ae62-0c3f3333e6c6 None None] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:18:32.494035 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-d24fdfb2-58ff-4f54-ae62-0c3f3333e6c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:18:32.539501 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aacb85c-a12b-4673-8abb-5d9d7e72000d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:18:32.540539 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aacb85c-a12b-4673-8abb-5d9d7e72000d admin admin] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:18:32.541072 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aacb85c-a12b-4673-8abb-5d9d7e72000d admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:18:32.541557 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aacb85c-a12b-4673-8abb-5d9d7e72000d admin admin] PATH_INFO: `/v3/users` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:18:32.542826 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aacb85c-a12b-4673-8abb-5d9d7e72000d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:18:32.546443 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aacb85c-a12b-4673-8abb-5d9d7e72000d admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:18:32.593607 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24fdfb2-58ff-4f54-ae62-0c3f3333e6c6 None None] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:18:32.598129 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0b9b2a-bc22-4e2e-87c7-fb67b737229f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a00b3fa7497456895e5e0bf105c4fc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd70d25bf69314820b8aaecb65576978a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:18:32.599361 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0b9b2a-bc22-4e2e-87c7-fb67b737229f service neutron] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:18:32.599875 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0b9b2a-bc22-4e2e-87c7-fb67b737229f service neutron] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:18:32.600358 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0b9b2a-bc22-4e2e-87c7-fb67b737229f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:18:32.626926 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0b9b2a-bc22-4e2e-87c7-fb67b737229f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:18:32.629266 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0b9b2a-bc22-4e2e-87c7-fb67b737229f service neutron] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:18:32.653466 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 893/1806] 10.222.0.23 () {60 vars in 1365 bytes} [Thu May 19 12:18:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:18:32.653466 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0b9b2a-bc22-4e2e-87c7-fb67b737229f service neutron] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:18:32.669659 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0b9b2a-bc22-4e2e-87c7-fb67b737229f service neutron] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:18:32.678481 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb825d7d-37d8-48f1-8d87-6b685ecbd42c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:18:32.679755 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb825d7d-37d8-48f1-8d87-6b685ecbd42c admin admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:18:32.680364 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb825d7d-37d8-48f1-8d87-6b685ecbd42c admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:18:32.681347 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb825d7d-37d8-48f1-8d87-6b685ecbd42c admin admin] PATH_INFO: `/v3/domains` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:18:32.683421 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb825d7d-37d8-48f1-8d87-6b685ecbd42c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:18:32.685050 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb825d7d-37d8-48f1-8d87-6b685ecbd42c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:18:32.690018 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb825d7d-37d8-48f1-8d87-6b685ecbd42c admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:18:32.708231 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=98,keystone:DELETE=1,keystone:INSERT=1 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:18:32.715828 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 894/1807] 10.222.0.23 () {58 vars in 1152 bytes} [Thu May 19 12:18:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:18:32.723663 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb825d7d-37d8-48f1-8d87-6b685ecbd42c admin admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:18:32.750792 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb825d7d-37d8-48f1-8d87-6b685ecbd42c admin admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:18:32.756928 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6fe953-cbbe-4957-8db6-639df396579c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:18:32.758593 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6fe953-cbbe-4957-8db6-639df396579c admin admin] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:18:32.759239 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6fe953-cbbe-4957-8db6-639df396579c admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:18:32.759239 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6fe953-cbbe-4957-8db6-639df396579c admin admin] PATH_INFO: `/v3/projects` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:18:32.760133 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6fe953-cbbe-4957-8db6-639df396579c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:18:32.762683 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6fe953-cbbe-4957-8db6-639df396579c admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:18:32.795890 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 914/1808] 10.222.0.23 () {60 vars in 1143 bytes} [Thu May 19 12:18:32 2022] POST /identity/v3/users => generated 292 bytes in 411 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 12:18:32.801950 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aacb85c-a12b-4673-8abb-5d9d7e72000d admin admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:18:32.828087 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aacb85c-a12b-4673-8abb-5d9d7e72000d admin admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:18:32.837385 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6a095e-110b-47ac-83a6-4da9a2720c3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:18:32.838554 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6a095e-110b-47ac-83a6-4da9a2720c3a admin admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:18:32.839270 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6a095e-110b-47ac-83a6-4da9a2720c3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:18:32.839989 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6a095e-110b-47ac-83a6-4da9a2720c3a admin admin] PATH_INFO: `/v3/roles` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:18:32.841691 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6a095e-110b-47ac-83a6-4da9a2720c3a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:18:32.845011 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6a095e-110b-47ac-83a6-4da9a2720c3a admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:18:32.886654 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 915/1809] 10.222.0.23 () {58 vars in 1121 bytes} [Thu May 19 12:18:32 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 19 12:18:32.888676 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6a095e-110b-47ac-83a6-4da9a2720c3a admin admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:18:32.909313 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6a095e-110b-47ac-83a6-4da9a2720c3a admin admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:18:32.936771 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c2a917-6b4f-47da-9520-d7c606c25f69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:18:32.938702 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c2a917-6b4f-47da-9520-d7c606c25f69 admin admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:18:32.939296 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c2a917-6b4f-47da-9520-d7c606c25f69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:18:32.939855 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c2a917-6b4f-47da-9520-d7c606c25f69 admin admin] PATH_INFO: `/v3/roles` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:18:32.942106 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c2a917-6b4f-47da-9520-d7c606c25f69 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:18:32.946907 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c2a917-6b4f-47da-9520-d7c606c25f69 admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:18:32.975849 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 916/1810] 10.222.0.23 () {58 vars in 1121 bytes} [Thu May 19 12:18:32 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 12:18:32.979979 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 895/1811] 10.222.0.23 () {60 vars in 1152 bytes} [Thu May 19 12:18:32 2022] POST /identity/v3/projects => generated 381 bytes in 260 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 12:18:32.985769 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6fe953-cbbe-4957-8db6-639df396579c admin admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:18:32.986064 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c2a917-6b4f-47da-9520-d7c606c25f69 admin admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:18:33.032940 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c2a917-6b4f-47da-9520-d7c606c25f69 admin admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:18:33.080138 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:18:33.081876 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6fe953-cbbe-4957-8db6-639df396579c admin admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:18:33.090678 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8338ebe-f3aa-4b6f-ab06-b9330090cc56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:18:33.091583 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8338ebe-f3aa-4b6f-ab06-b9330090cc56 admin admin] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:18:33.092018 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8338ebe-f3aa-4b6f-ab06-b9330090cc56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:18:33.092557 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8338ebe-f3aa-4b6f-ab06-b9330090cc56 admin admin] PATH_INFO: `/v3/users` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:18:33.093484 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8338ebe-f3aa-4b6f-ab06-b9330090cc56 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:18:33.102477 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8338ebe-f3aa-4b6f-ab06-b9330090cc56 admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:18:33.236991 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e848b2ba-5c3f-40be-b46b-82a3f3b0e9ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:18:33.237747 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e848b2ba-5c3f-40be-b46b-82a3f3b0e9ac admin admin] REQUEST_METHOD: `PUT` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:18:33.239133 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e848b2ba-5c3f-40be-b46b-82a3f3b0e9ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:18:33.239590 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e848b2ba-5c3f-40be-b46b-82a3f3b0e9ac admin admin] PATH_INFO: `/v3/projects/3f2a1b6c9bd74b8c825d17d4362c01e9/users/c4ca456011124b7aacca438fc74feee9/roles/259eb400cdb4439db72d14456ae9cd08` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:18:33.252225 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 896/1812] 10.222.0.23 () {60 vars in 1143 bytes} [Thu May 19 12:18:32 2022] POST /identity/v3/users => generated 291 bytes in 266 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 12:18:33.256509 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8338ebe-f3aa-4b6f-ab06-b9330090cc56 admin admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:18:33.323145 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8338ebe-f3aa-4b6f-ab06-b9330090cc56 admin admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:18:33.336247 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e848b2ba-5c3f-40be-b46b-82a3f3b0e9ac admin admin] RBAC: Authorizing `identity:create_grant(project_id=3f2a1b6c9bd74b8c825d17d4362c01e9, user_id=c4ca456011124b7aacca438fc74feee9, role_id=259eb400cdb4439db72d14456ae9cd08)` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:18:33.342440 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e848b2ba-5c3f-40be-b46b-82a3f3b0e9ac admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:18:33.343626 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af8b085-cca5-4c3a-94f9-fd34421b6660 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:18:33.344520 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af8b085-cca5-4c3a-94f9-fd34421b6660 admin admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:18:33.344970 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af8b085-cca5-4c3a-94f9-fd34421b6660 admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:18:33.345389 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af8b085-cca5-4c3a-94f9-fd34421b6660 admin admin] PATH_INFO: `/v3/roles` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:18:33.346347 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af8b085-cca5-4c3a-94f9-fd34421b6660 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:18:33.348973 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af8b085-cca5-4c3a-94f9-fd34421b6660 admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:18:33.363922 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 897/1813] 10.222.0.23 () {58 vars in 1121 bytes} [Thu May 19 12:18:33 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 12:18:33.369456 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af8b085-cca5-4c3a-94f9-fd34421b6660 admin admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:18:33.380990 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af8b085-cca5-4c3a-94f9-fd34421b6660 admin admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:18:33.399220 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042abb7e-9700-40fa-ae6a-2a19f7e2a4d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:18:33.400517 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042abb7e-9700-40fa-ae6a-2a19f7e2a4d6 admin admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:18:33.401342 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042abb7e-9700-40fa-ae6a-2a19f7e2a4d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:18:33.402047 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042abb7e-9700-40fa-ae6a-2a19f7e2a4d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:18:33.405440 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042abb7e-9700-40fa-ae6a-2a19f7e2a4d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:18:33.412127 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042abb7e-9700-40fa-ae6a-2a19f7e2a4d6 admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:18:33.440366 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 898/1814] 10.222.0.23 () {58 vars in 1121 bytes} [Thu May 19 12:18:33 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 12:18:33.447970 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042abb7e-9700-40fa-ae6a-2a19f7e2a4d6 admin admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:18:33.484308 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 917/1815] 10.222.0.23 () {60 vars in 1482 bytes} [Thu May 19 12:18:32 2022] PUT /identity/v3/projects/3f2a1b6c9bd74b8c825d17d4362c01e9/users/c4ca456011124b7aacca438fc74feee9/roles/259eb400cdb4439db72d14456ae9cd08 => generated 0 bytes in 504 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 12:18:33.486197 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042abb7e-9700-40fa-ae6a-2a19f7e2a4d6 admin admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:18:33.499278 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f0e02b-c567-4fad-9617-1edd1d8c74b1 None None] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:18:33.499278 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f0e02b-c567-4fad-9617-1edd1d8c74b1 None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:18:33.499278 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f0e02b-c567-4fad-9617-1edd1d8c74b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:18:33.556079 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.auth.core [None req-77f0e02b-c567-4fad-9617-1edd1d8c74b1 None None] MFA Rules not processed for user `c4ca456011124b7aacca438fc74feee9`. Rule list: `[]` (Enabled: `True`). {{(pid=80540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:18:33.557820 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-77f0e02b-c567-4fad-9617-1edd1d8c74b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:18:33.679378 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee47e49-3cfb-42c5-b1b3-c1ad842e36c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:18:33.681289 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee47e49-3cfb-42c5-b1b3-c1ad842e36c6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:18:33.681865 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee47e49-3cfb-42c5-b1b3-c1ad842e36c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:18:33.682363 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee47e49-3cfb-42c5-b1b3-c1ad842e36c6 admin admin] PATH_INFO: `/v3/projects/0d86579d965f4b58b343e87d2e4960f9/users/facbc625174c435591bdad17e472a4fb/roles/259eb400cdb4439db72d14456ae9cd08` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:18:33.720647 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 918/1816] 10.222.0.23 () {58 vars in 957 bytes} [Thu May 19 12:18:33 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:18:33.722729 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f0e02b-c567-4fad-9617-1edd1d8c74b1 None None] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:18:33.724768 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-77f0e02b-c567-4fad-9617-1edd1d8c74b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:18:33.729364 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee47e49-3cfb-42c5-b1b3-c1ad842e36c6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0d86579d965f4b58b343e87d2e4960f9, user_id=facbc625174c435591bdad17e472a4fb, role_id=259eb400cdb4439db72d14456ae9cd08)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:18:33.735693 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee47e49-3cfb-42c5-b1b3-c1ad842e36c6 admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:18:33.742057 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f0e02b-c567-4fad-9617-1edd1d8c74b1 None None] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:18:33.748250 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a7486d-a753-4d61-9ce8-ead95f2cba69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:18:33.749217 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a7486d-a753-4d61-9ce8-ead95f2cba69 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:18:33.749501 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a7486d-a753-4d61-9ce8-ead95f2cba69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:18:33.749780 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a7486d-a753-4d61-9ce8-ead95f2cba69 admin admin] PATH_INFO: `/v3/projects/3f82f9f8661b44709371c4d574b4de0a` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:18:33.751118 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a7486d-a753-4d61-9ce8-ead95f2cba69 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3f82f9f8661b44709371c4d574b4de0a)` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:18:33.753279 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a7486d-a753-4d61-9ce8-ead95f2cba69 admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:18:33.842618 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 899/1817] 10.222.0.23 () {60 vars in 1482 bytes} [Thu May 19 12:18:33 2022] PUT /identity/v3/projects/0d86579d965f4b58b343e87d2e4960f9/users/facbc625174c435591bdad17e472a4fb/roles/259eb400cdb4439db72d14456ae9cd08 => generated 0 bytes in 396 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 12:18:33.843462 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee47e49-3cfb-42c5-b1b3-c1ad842e36c6 admin admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:18:33.877856 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:18:33.940456 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee47e49-3cfb-42c5-b1b3-c1ad842e36c6 admin admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:18:33.952184 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-f3a7486d-a753-4d61-9ce8-ead95f2cba69 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3f82f9f8661b44709371c4d574b4de0a', 'request_id': 'req-f3a7486d-a753-4d61-9ce8-ead95f2cba69'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:18:33.962567 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-f3a7486d-a753-4d61-9ce8-ead95f2cba69 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3f82f9f8661b44709371c4d574b4de0a', 'request_id': 'req-f3a7486d-a753-4d61-9ce8-ead95f2cba69'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:18:33.973033 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-f3a7486d-a753-4d61-9ce8-ead95f2cba69 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3f82f9f8661b44709371c4d574b4de0a', 'request_id': 'req-f3a7486d-a753-4d61-9ce8-ead95f2cba69'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:18:34.058066 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd6b8db-ba6b-4999-abca-e313f4ac166b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:18:34.059280 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd6b8db-ba6b-4999-abca-e313f4ac166b admin admin] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:18:34.059743 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd6b8db-ba6b-4999-abca-e313f4ac166b admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:18:34.060291 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd6b8db-ba6b-4999-abca-e313f4ac166b admin admin] PATH_INFO: `/v3/projects` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:18:34.061437 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd6b8db-ba6b-4999-abca-e313f4ac166b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:18:34.064406 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd6b8db-ba6b-4999-abca-e313f4ac166b admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:18:34.105372 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-f3a7486d-a753-4d61-9ce8-ead95f2cba69 admin admin] The token cache is being invalidate because project 3f82f9f8661b44709371c4d574b4de0a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80540) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 12:18:34.105886 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-f3a7486d-a753-4d61-9ce8-ead95f2cba69 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:18:34.107067 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-f3a7486d-a753-4d61-9ce8-ead95f2cba69 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:18:34.110184 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 919/1818] 10.222.0.23 () {58 vars in 1232 bytes} [Thu May 19 12:18:33 2022] DELETE /identity/v3/projects/3f82f9f8661b44709371c4d574b4de0a => generated 0 bytes in 389 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:18:34.116319 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddbcee9-4aa1-40a4-8c21-42d19bbb75b4 None None] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:18:34.117608 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddbcee9-4aa1-40a4-8c21-42d19bbb75b4 None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:18:34.119080 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddbcee9-4aa1-40a4-8c21-42d19bbb75b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:18:34.166473 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:18:34.183410 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.auth.core [None req-dddbcee9-4aa1-40a4-8c21-42d19bbb75b4 None None] MFA Rules not processed for user `facbc625174c435591bdad17e472a4fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:18:34.185572 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-dddbcee9-4aa1-40a4-8c21-42d19bbb75b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:18:34.316209 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 920/1819] 10.222.0.23 () {58 vars in 957 bytes} [Thu May 19 12:18:34 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:18:34.317276 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dddbcee9-4aa1-40a4-8c21-42d19bbb75b4 None None] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:18:34.319069 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-dddbcee9-4aa1-40a4-8c21-42d19bbb75b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:18:34.338274 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dddbcee9-4aa1-40a4-8c21-42d19bbb75b4 None None] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:18:34.343780 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8be489-9887-4b04-868d-ed6dda4bd1f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:18:34.344894 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8be489-9887-4b04-868d-ed6dda4bd1f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:18:34.345245 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8be489-9887-4b04-868d-ed6dda4bd1f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:18:34.345569 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8be489-9887-4b04-868d-ed6dda4bd1f3 admin admin] PATH_INFO: `/v3/users/4174b35cc2ee4dd880b1e978de360413` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:18:34.347149 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8be489-9887-4b04-868d-ed6dda4bd1f3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4174b35cc2ee4dd880b1e978de360413)` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:18:34.349568 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8be489-9887-4b04-868d-ed6dda4bd1f3 admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:18:34.465497 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 900/1820] 10.222.0.23 () {60 vars in 1152 bytes} [Thu May 19 12:18:33 2022] POST /identity/v3/projects => generated 387 bytes in 622 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 12:18:34.467823 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd6b8db-ba6b-4999-abca-e313f4ac166b admin admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:18:34.470165 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-6bd6b8db-ba6b-4999-abca-e313f4ac166b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:18:34.575004 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd6b8db-ba6b-4999-abca-e313f4ac166b admin admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:18:34.582391 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75f257d-382b-477d-b8b6-266a51d88c85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:18:34.583425 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75f257d-382b-477d-b8b6-266a51d88c85 admin admin] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:18:34.584000 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75f257d-382b-477d-b8b6-266a51d88c85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:18:34.584250 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75f257d-382b-477d-b8b6-266a51d88c85 admin admin] PATH_INFO: `/v3/projects` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:18:34.585005 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75f257d-382b-477d-b8b6-266a51d88c85 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:18:34.588309 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75f257d-382b-477d-b8b6-266a51d88c85 admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:18:34.707711 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-4b8be489-9887-4b04-868d-ed6dda4bd1f3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4174b35cc2ee4dd880b1e978de360413'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:18:34.740666 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 901/1821] 10.222.0.23 () {60 vars in 1152 bytes} [Thu May 19 12:18:34 2022] POST /identity/v3/projects => generated 379 bytes in 275 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 12:18:34.742485 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75f257d-382b-477d-b8b6-266a51d88c85 admin admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:18:34.756973 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-4b8be489-9887-4b04-868d-ed6dda4bd1f3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4174b35cc2ee4dd880b1e978de360413'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:18:34.757245 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-b75f257d-382b-477d-b8b6-266a51d88c85 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:18:34.760569 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-4b8be489-9887-4b04-868d-ed6dda4bd1f3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4174b35cc2ee4dd880b1e978de360413'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:18:34.834518 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:18:34.862469 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-4b8be489-9887-4b04-868d-ed6dda4bd1f3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4174b35cc2ee4dd880b1e978de360413'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:18:34.866126 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75f257d-382b-477d-b8b6-266a51d88c85 admin admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:18:34.873530 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca954d9b-3bef-454d-b3f6-9e87a3616a71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:18:34.874638 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca954d9b-3bef-454d-b3f6-9e87a3616a71 admin admin] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:18:34.875131 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca954d9b-3bef-454d-b3f6-9e87a3616a71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:18:34.875457 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-4b8be489-9887-4b04-868d-ed6dda4bd1f3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4174b35cc2ee4dd880b1e978de360413'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:18:34.875604 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca954d9b-3bef-454d-b3f6-9e87a3616a71 admin admin] PATH_INFO: `/v3/users` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:18:34.876857 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca954d9b-3bef-454d-b3f6-9e87a3616a71 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:18:34.885268 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca954d9b-3bef-454d-b3f6-9e87a3616a71 admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:18:34.887544 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 921/1822] 10.222.0.23 () {58 vars in 1223 bytes} [Thu May 19 12:18:34 2022] DELETE /identity/v3/users/4174b35cc2ee4dd880b1e978de360413 => generated 0 bytes in 571 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:18:34.888934 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8be489-9887-4b04-868d-ed6dda4bd1f3 admin admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:18:34.891892 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-4b8be489-9887-4b04-868d-ed6dda4bd1f3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:18:34.981657 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 902/1823] 10.222.0.23 () {60 vars in 1143 bytes} [Thu May 19 12:18:34 2022] POST /identity/v3/users => generated 300 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 12:18:34.983896 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca954d9b-3bef-454d-b3f6-9e87a3616a71 admin admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:18:34.986838 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-ca954d9b-3bef-454d-b3f6-9e87a3616a71 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:18:35.001916 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8be489-9887-4b04-868d-ed6dda4bd1f3 admin admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:18:35.010540 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d683c98-9cbd-4ac1-9eb7-c923483ec78b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:18:35.010947 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d683c98-9cbd-4ac1-9eb7-c923483ec78b admin admin] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:18:35.011234 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d683c98-9cbd-4ac1-9eb7-c923483ec78b admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:18:35.011518 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d683c98-9cbd-4ac1-9eb7-c923483ec78b admin admin] PATH_INFO: `/v3/users` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:18:35.012393 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d683c98-9cbd-4ac1-9eb7-c923483ec78b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:18:35.014337 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d683c98-9cbd-4ac1-9eb7-c923483ec78b admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:18:35.017598 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca954d9b-3bef-454d-b3f6-9e87a3616a71 admin admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:18:35.024481 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf076fa6-ba7f-4ff7-9dda-0fd376e6ad30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:18:35.025410 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf076fa6-ba7f-4ff7-9dda-0fd376e6ad30 admin admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:18:35.025767 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf076fa6-ba7f-4ff7-9dda-0fd376e6ad30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:18:35.026049 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf076fa6-ba7f-4ff7-9dda-0fd376e6ad30 admin admin] PATH_INFO: `/v3/roles` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:18:35.026826 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf076fa6-ba7f-4ff7-9dda-0fd376e6ad30 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:18:35.028497 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf076fa6-ba7f-4ff7-9dda-0fd376e6ad30 admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:18:35.041095 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 903/1824] 10.222.0.23 () {58 vars in 1121 bytes} [Thu May 19 12:18:34 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 12:18:35.051822 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf076fa6-ba7f-4ff7-9dda-0fd376e6ad30 admin admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:18:35.060258 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf076fa6-ba7f-4ff7-9dda-0fd376e6ad30 admin admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:18:35.066542 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc295a5-2d6d-4134-abf0-a9db1d4a9aea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:18:35.067542 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc295a5-2d6d-4134-abf0-a9db1d4a9aea admin admin] REQUEST_METHOD: `PUT` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:18:35.067943 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc295a5-2d6d-4134-abf0-a9db1d4a9aea admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:18:35.068282 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc295a5-2d6d-4134-abf0-a9db1d4a9aea admin admin] PATH_INFO: `/v3/projects/238a310adf454901970f67a73b78c4dc/users/df27066a0d674d7ca3c0b2b22f61f2dd/roles/9db5671c65d3414ea2682afa2ccf1403` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:18:35.089540 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc295a5-2d6d-4134-abf0-a9db1d4a9aea admin admin] RBAC: Authorizing `identity:create_grant(project_id=238a310adf454901970f67a73b78c4dc, user_id=df27066a0d674d7ca3c0b2b22f61f2dd, role_id=9db5671c65d3414ea2682afa2ccf1403)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:18:35.094624 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc295a5-2d6d-4134-abf0-a9db1d4a9aea admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:18:35.147280 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:18:35.155218 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 922/1825] 10.222.0.23 () {60 vars in 1143 bytes} [Thu May 19 12:18:34 2022] POST /identity/v3/users => generated 296 bytes in 267 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 12:18:35.160487 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d683c98-9cbd-4ac1-9eb7-c923483ec78b admin admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:18:35.176739 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d683c98-9cbd-4ac1-9eb7-c923483ec78b admin admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:18:35.183005 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083f980f-db72-4b5a-a4ef-7060495099fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:18:35.183702 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083f980f-db72-4b5a-a4ef-7060495099fa admin admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:18:35.184054 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083f980f-db72-4b5a-a4ef-7060495099fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:18:35.184442 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083f980f-db72-4b5a-a4ef-7060495099fa admin admin] PATH_INFO: `/v3/roles` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:18:35.185162 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083f980f-db72-4b5a-a4ef-7060495099fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:18:35.186846 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083f980f-db72-4b5a-a4ef-7060495099fa admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:18:35.198701 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 923/1826] 10.222.0.23 () {58 vars in 1121 bytes} [Thu May 19 12:18:35 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 19 12:18:35.203272 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083f980f-db72-4b5a-a4ef-7060495099fa admin admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:18:35.217414 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083f980f-db72-4b5a-a4ef-7060495099fa admin admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:18:35.222433 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563828ec-1c57-4855-b6e3-93ace261fcc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:18:35.223740 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563828ec-1c57-4855-b6e3-93ace261fcc3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:18:35.224069 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563828ec-1c57-4855-b6e3-93ace261fcc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:18:35.224362 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563828ec-1c57-4855-b6e3-93ace261fcc3 admin admin] PATH_INFO: `/v3/projects/633bf351dd7c494e9bbabd2704f0e517/users/4f7e7b637163431689dc60301661d398/roles/9db5671c65d3414ea2682afa2ccf1403` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:18:35.245369 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563828ec-1c57-4855-b6e3-93ace261fcc3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=633bf351dd7c494e9bbabd2704f0e517, user_id=4f7e7b637163431689dc60301661d398, role_id=9db5671c65d3414ea2682afa2ccf1403)` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:18:35.248154 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563828ec-1c57-4855-b6e3-93ace261fcc3 admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:18:35.445282 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 904/1827] 10.222.0.23 () {60 vars in 1482 bytes} [Thu May 19 12:18:35 2022] PUT /identity/v3/projects/238a310adf454901970f67a73b78c4dc/users/df27066a0d674d7ca3c0b2b22f61f2dd/roles/9db5671c65d3414ea2682afa2ccf1403 => generated 0 bytes in 395 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:18:35.446532 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 924/1828] 10.222.0.23 () {60 vars in 1482 bytes} [Thu May 19 12:18:35 2022] PUT /identity/v3/projects/633bf351dd7c494e9bbabd2704f0e517/users/4f7e7b637163431689dc60301661d398/roles/9db5671c65d3414ea2682afa2ccf1403 => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 12:18:35.446707 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc295a5-2d6d-4134-abf0-a9db1d4a9aea admin admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:18:35.452741 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563828ec-1c57-4855-b6e3-93ace261fcc3 admin admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:18:35.556803 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563828ec-1c57-4855-b6e3-93ace261fcc3 admin admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:18:35.561322 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc295a5-2d6d-4134-abf0-a9db1d4a9aea admin admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:18:35.562249 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2985adf9-ed42-4973-b23d-eeaebec7b190 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:18:35.562786 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2985adf9-ed42-4973-b23d-eeaebec7b190 admin admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:18:35.563190 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2985adf9-ed42-4973-b23d-eeaebec7b190 admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:18:35.563578 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2985adf9-ed42-4973-b23d-eeaebec7b190 admin admin] PATH_INFO: `/v3/roles` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:18:35.564581 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2985adf9-ed42-4973-b23d-eeaebec7b190 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:18:35.567575 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2985adf9-ed42-4973-b23d-eeaebec7b190 admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:18:35.567805 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23bb127-f8d0-4446-8ac7-f419e959da2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:18:35.568288 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23bb127-f8d0-4446-8ac7-f419e959da2f admin admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:18:35.569115 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23bb127-f8d0-4446-8ac7-f419e959da2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:18:35.569806 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23bb127-f8d0-4446-8ac7-f419e959da2f admin admin] PATH_INFO: `/v3/roles` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:18:35.571750 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23bb127-f8d0-4446-8ac7-f419e959da2f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:18:35.573556 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23bb127-f8d0-4446-8ac7-f419e959da2f admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:18:35.580866 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 925/1829] 10.222.0.23 () {58 vars in 1121 bytes} [Thu May 19 12:18:35 2022] GET /identity/v3/roles => generated 1420 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 12:18:35.584820 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2985adf9-ed42-4973-b23d-eeaebec7b190 admin admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:18:35.597180 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 905/1830] 10.222.0.23 () {58 vars in 1121 bytes} [Thu May 19 12:18:35 2022] GET /identity/v3/roles => generated 1420 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 12:18:35.604028 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23bb127-f8d0-4446-8ac7-f419e959da2f admin admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:18:35.606218 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2985adf9-ed42-4973-b23d-eeaebec7b190 admin admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:18:35.612082 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e544cc2-274f-4a4a-a405-4e1bdcd39118 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:18:35.613579 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e544cc2-274f-4a4a-a405-4e1bdcd39118 admin admin] REQUEST_METHOD: `PUT` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:18:35.614086 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e544cc2-274f-4a4a-a405-4e1bdcd39118 admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:18:35.614478 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e544cc2-274f-4a4a-a405-4e1bdcd39118 admin admin] PATH_INFO: `/v3/projects/633bf351dd7c494e9bbabd2704f0e517/users/4f7e7b637163431689dc60301661d398/roles/9db5671c65d3414ea2682afa2ccf1403` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:18:35.617670 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23bb127-f8d0-4446-8ac7-f419e959da2f admin admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:18:35.617846 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e544cc2-274f-4a4a-a405-4e1bdcd39118 admin admin] RBAC: Authorizing `identity:create_grant(project_id=633bf351dd7c494e9bbabd2704f0e517, user_id=4f7e7b637163431689dc60301661d398, role_id=9db5671c65d3414ea2682afa2ccf1403)` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:18:35.620627 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e544cc2-274f-4a4a-a405-4e1bdcd39118 admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:18:35.623771 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391ab34e-c7bc-46e1-bcef-9accebf4f73d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:18:35.625123 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391ab34e-c7bc-46e1-bcef-9accebf4f73d admin admin] REQUEST_METHOD: `PUT` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:18:35.641515 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391ab34e-c7bc-46e1-bcef-9accebf4f73d admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:18:35.641515 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391ab34e-c7bc-46e1-bcef-9accebf4f73d admin admin] PATH_INFO: `/v3/projects/238a310adf454901970f67a73b78c4dc/users/df27066a0d674d7ca3c0b2b22f61f2dd/roles/9db5671c65d3414ea2682afa2ccf1403` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:18:35.641515 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391ab34e-c7bc-46e1-bcef-9accebf4f73d admin admin] RBAC: Authorizing `identity:create_grant(project_id=238a310adf454901970f67a73b78c4dc, user_id=df27066a0d674d7ca3c0b2b22f61f2dd, role_id=9db5671c65d3414ea2682afa2ccf1403)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:18:35.641515 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391ab34e-c7bc-46e1-bcef-9accebf4f73d admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:18:35.642141 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 926/1831] 10.222.0.23 () {60 vars in 1482 bytes} [Thu May 19 12:18:35 2022] PUT /identity/v3/projects/633bf351dd7c494e9bbabd2704f0e517/users/4f7e7b637163431689dc60301661d398/roles/9db5671c65d3414ea2682afa2ccf1403 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 12:18:35.645066 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 906/1832] 10.222.0.23 () {60 vars in 1482 bytes} [Thu May 19 12:18:35 2022] PUT /identity/v3/projects/238a310adf454901970f67a73b78c4dc/users/df27066a0d674d7ca3c0b2b22f61f2dd/roles/9db5671c65d3414ea2682afa2ccf1403 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:18:35.651210 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c62ffa2-46e3-48aa-b8f6-f2d4bd6a2ae3 None None] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:18:35.651616 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c62ffa2-46e3-48aa-b8f6-f2d4bd6a2ae3 None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:18:35.651894 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c62ffa2-46e3-48aa-b8f6-f2d4bd6a2ae3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:18:35.665315 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eaea0ae-0e56-4451-8e9f-5b4871ad1ddb None None] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:18:35.665315 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eaea0ae-0e56-4451-8e9f-5b4871ad1ddb None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:18:35.665776 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eaea0ae-0e56-4451-8e9f-5b4871ad1ddb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:18:35.698712 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.auth.core [None req-3c62ffa2-46e3-48aa-b8f6-f2d4bd6a2ae3 None None] MFA Rules not processed for user `4f7e7b637163431689dc60301661d398`. Rule list: `[]` (Enabled: `True`). {{(pid=80540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:18:35.700128 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-3c62ffa2-46e3-48aa-b8f6-f2d4bd6a2ae3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:18:35.743168 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.auth.core [None req-7eaea0ae-0e56-4451-8e9f-5b4871ad1ddb None None] MFA Rules not processed for user `df27066a0d674d7ca3c0b2b22f61f2dd`. Rule list: `[]` (Enabled: `True`). {{(pid=80539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:18:35.746279 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-7eaea0ae-0e56-4451-8e9f-5b4871ad1ddb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:18:35.856862 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 927/1833] 10.222.0.23 () {58 vars in 957 bytes} [Thu May 19 12:18:35 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 12:18:35.859876 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 907/1834] 10.222.0.23 () {58 vars in 957 bytes} [Thu May 19 12:18:35 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:18:35.876678 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c62ffa2-46e3-48aa-b8f6-f2d4bd6a2ae3 None None] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:18:35.877777 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-3c62ffa2-46e3-48aa-b8f6-f2d4bd6a2ae3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:18:35.920550 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eaea0ae-0e56-4451-8e9f-5b4871ad1ddb None None] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:18:35.932559 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-7eaea0ae-0e56-4451-8e9f-5b4871ad1ddb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:18:35.939809 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:18:35.982628 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c62ffa2-46e3-48aa-b8f6-f2d4bd6a2ae3 None None] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:18:35.983105 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:18:35.997421 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7014611-866f-4e8f-b56f-14f03aa696f8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8800254c0e784e42a791fb9abc274d79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd70d25bf69314820b8aaecb65576978a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:18:35.998553 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7014611-866f-4e8f-b56f-14f03aa696f8 service cinder] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:18:35.998855 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7014611-866f-4e8f-b56f-14f03aa696f8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:18:36.000344 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7014611-866f-4e8f-b56f-14f03aa696f8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:18:36.013840 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7014611-866f-4e8f-b56f-14f03aa696f8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:18:36.016103 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7014611-866f-4e8f-b56f-14f03aa696f8 service cinder] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:18:36.056961 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 928/1835] 10.222.0.23 () {60 vars in 1365 bytes} [Thu May 19 12:18:35 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:18:36.058104 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7014611-866f-4e8f-b56f-14f03aa696f8 service cinder] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:18:36.063153 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-a7014611-866f-4e8f-b56f-14f03aa696f8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:18:36.098206 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7014611-866f-4e8f-b56f-14f03aa696f8 service cinder] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:18:36.103516 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9dfa38e-7255-4409-8042-bf564f2a947d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8800254c0e784e42a791fb9abc274d79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd70d25bf69314820b8aaecb65576978a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:18:36.104234 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9dfa38e-7255-4409-8042-bf564f2a947d service cinder] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:18:36.104529 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9dfa38e-7255-4409-8042-bf564f2a947d service cinder] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:18:36.104814 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9dfa38e-7255-4409-8042-bf564f2a947d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:18:36.118233 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9dfa38e-7255-4409-8042-bf564f2a947d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:18:36.120008 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9dfa38e-7255-4409-8042-bf564f2a947d service cinder] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:18:36.133718 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 929/1836] 10.222.0.23 () {60 vars in 1365 bytes} [Thu May 19 12:18:36 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 76 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:18:36.142191 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1194d3-edd1-48a4-b1fc-8af6bb06e340 None None] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:18:36.142580 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1194d3-edd1-48a4-b1fc-8af6bb06e340 None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:18:36.142806 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1194d3-edd1-48a4-b1fc-8af6bb06e340 None None] PATH_INFO: `/` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:18:36.144965 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 930/1837] 10.222.0.23 () {54 vars in 921 bytes} [Thu May 19 12:18:36 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 12:18:36.169136 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431f4112-5a68-4899-81b6-5ad6a7d8146d None None] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:18:36.169454 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431f4112-5a68-4899-81b6-5ad6a7d8146d None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:18:36.169798 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431f4112-5a68-4899-81b6-5ad6a7d8146d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:18:36.205992 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eaea0ae-0e56-4451-8e9f-5b4871ad1ddb None None] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:18:36.232857 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a116e3-1d48-4f80-b0d3-8b184acde488 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:18:36.232857 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a116e3-1d48-4f80-b0d3-8b184acde488 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:18:36.232857 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a116e3-1d48-4f80-b0d3-8b184acde488 admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:18:36.232857 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a116e3-1d48-4f80-b0d3-8b184acde488 admin admin] PATH_INFO: `/v3/projects/bdb86b1eb0cc48eeab60c78e68cb510f` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:18:36.233136 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.auth.core [None req-431f4112-5a68-4899-81b6-5ad6a7d8146d None None] MFA Rules not processed for user `34a98d75d84b4c67981377e0f26eca62`. Rule list: `[]` (Enabled: `True`). {{(pid=80540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:18:36.233136 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-431f4112-5a68-4899-81b6-5ad6a7d8146d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:18:36.233270 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a116e3-1d48-4f80-b0d3-8b184acde488 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bdb86b1eb0cc48eeab60c78e68cb510f)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:18:36.236866 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a116e3-1d48-4f80-b0d3-8b184acde488 admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:18:36.395892 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 931/1838] 10.222.0.23 () {58 vars in 1018 bytes} [Thu May 19 12:18:36 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:18:36.397153 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431f4112-5a68-4899-81b6-5ad6a7d8146d None None] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:18:36.430630 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-e0a116e3-1d48-4f80-b0d3-8b184acde488 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bdb86b1eb0cc48eeab60c78e68cb510f', 'request_id': 'req-e0a116e3-1d48-4f80-b0d3-8b184acde488'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:18:36.435885 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431f4112-5a68-4899-81b6-5ad6a7d8146d None None] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:18:36.441187 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-e0a116e3-1d48-4f80-b0d3-8b184acde488 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bdb86b1eb0cc48eeab60c78e68cb510f', 'request_id': 'req-e0a116e3-1d48-4f80-b0d3-8b184acde488'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:18:36.524624 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-e0a116e3-1d48-4f80-b0d3-8b184acde488 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bdb86b1eb0cc48eeab60c78e68cb510f', 'request_id': 'req-e0a116e3-1d48-4f80-b0d3-8b184acde488'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:18:36.545856 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-e0a116e3-1d48-4f80-b0d3-8b184acde488 admin admin] The token cache is being invalidate because project bdb86b1eb0cc48eeab60c78e68cb510f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80539) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 12:18:36.546302 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-e0a116e3-1d48-4f80-b0d3-8b184acde488 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:18:36.547126 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-e0a116e3-1d48-4f80-b0d3-8b184acde488 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:18:36.548911 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 908/1839] 10.222.0.23 () {58 vars in 1232 bytes} [Thu May 19 12:18:35 2022] DELETE /identity/v3/projects/bdb86b1eb0cc48eeab60c78e68cb510f => generated 0 bytes in 634 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:18:36.549741 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a116e3-1d48-4f80-b0d3-8b184acde488 admin admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:18:36.551122 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-e0a116e3-1d48-4f80-b0d3-8b184acde488 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:18:36.562465 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e0583f-a9ee-4016-91b6-187404b0ffdf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8800254c0e784e42a791fb9abc274d79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd70d25bf69314820b8aaecb65576978a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:18:36.565849 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e0583f-a9ee-4016-91b6-187404b0ffdf service cinder] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:18:36.565849 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e0583f-a9ee-4016-91b6-187404b0ffdf service cinder] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:18:36.565849 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e0583f-a9ee-4016-91b6-187404b0ffdf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:18:36.566141 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-e9e0583f-a9ee-4016-91b6-187404b0ffdf service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:18:36.658789 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a116e3-1d48-4f80-b0d3-8b184acde488 admin admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:18:36.667402 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697033c8-175c-4776-aa71-3875afcf991b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34a98d75d84b4c67981377e0f26eca62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd70d25bf69314820b8aaecb65576978a', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:18:36.668177 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697033c8-175c-4776-aa71-3875afcf991b service nova] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:18:36.668466 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697033c8-175c-4776-aa71-3875afcf991b service nova] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:18:36.669017 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697033c8-175c-4776-aa71-3875afcf991b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:18:36.670270 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e0583f-a9ee-4016-91b6-187404b0ffdf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:18:36.670837 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-697033c8-175c-4776-aa71-3875afcf991b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:18:36.672134 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e0583f-a9ee-4016-91b6-187404b0ffdf service cinder] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:18:36.682851 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 932/1840] 10.222.0.23 () {60 vars in 1365 bytes} [Thu May 19 12:18:36 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 286 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:18:36.684478 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53a6fdc-510d-4d42-83b0-a6e7f7c832d3 None None] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:18:36.684639 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53a6fdc-510d-4d42-83b0-a6e7f7c832d3 None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:18:36.684960 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53a6fdc-510d-4d42-83b0-a6e7f7c832d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:18:36.686571 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-697033c8-175c-4776-aa71-3875afcf991b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:18:36.689039 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-697033c8-175c-4776-aa71-3875afcf991b service nova] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:18:36.715400 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 909/1841] 10.222.0.23 () {60 vars in 1365 bytes} [Thu May 19 12:18:36 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:18:36.717369 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.auth.core [None req-d53a6fdc-510d-4d42-83b0-a6e7f7c832d3 None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:18:36.719473 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-d53a6fdc-510d-4d42-83b0-a6e7f7c832d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:18:36.727574 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:18:36.816128 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 933/1842] 10.222.0.23 () {58 vars in 957 bytes} [Thu May 19 12:18:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:18:36.825907 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77fb1da-e02e-40f6-9c96-4ad2db5d4741 None None] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:18:36.826203 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77fb1da-e02e-40f6-9c96-4ad2db5d4741 None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:18:36.826773 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77fb1da-e02e-40f6-9c96-4ad2db5d4741 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:18:36.848019 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.auth.core [None req-a77fb1da-e02e-40f6-9c96-4ad2db5d4741 None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:18:36.849354 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-a77fb1da-e02e-40f6-9c96-4ad2db5d4741 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:18:36.858542 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 910/1843] 10.222.0.23 () {58 vars in 957 bytes} [Thu May 19 12:18:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:18:36.862874 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53a6fdc-510d-4d42-83b0-a6e7f7c832d3 None None] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:18:36.872901 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53a6fdc-510d-4d42-83b0-a6e7f7c832d3 None None] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:18:36.877692 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf25f19a-5cf0-468f-8b46-d9406640ebfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:18:36.878374 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf25f19a-5cf0-468f-8b46-d9406640ebfc admin admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:18:36.878665 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf25f19a-5cf0-468f-8b46-d9406640ebfc admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:18:36.878943 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf25f19a-5cf0-468f-8b46-d9406640ebfc admin admin] PATH_INFO: `/v3/domains` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:18:36.879691 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf25f19a-5cf0-468f-8b46-d9406640ebfc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:18:36.880035 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf25f19a-5cf0-468f-8b46-d9406640ebfc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:18:36.883071 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf25f19a-5cf0-468f-8b46-d9406640ebfc admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:18:36.902845 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 934/1844] 10.222.0.23 () {58 vars in 1152 bytes} [Thu May 19 12:18:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:18:36.908784 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77fb1da-e02e-40f6-9c96-4ad2db5d4741 None None] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:18:36.926337 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77fb1da-e02e-40f6-9c96-4ad2db5d4741 None None] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:18:36.931685 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3efe5e-d798-459c-9ccd-ba932710b09b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:18:36.932398 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3efe5e-d798-459c-9ccd-ba932710b09b admin admin] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:18:36.932676 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3efe5e-d798-459c-9ccd-ba932710b09b admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:18:36.932949 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3efe5e-d798-459c-9ccd-ba932710b09b admin admin] PATH_INFO: `/v3/projects` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:18:36.933730 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3efe5e-d798-459c-9ccd-ba932710b09b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:18:36.935594 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3efe5e-d798-459c-9ccd-ba932710b09b admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:18:36.957661 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:18:37.121830 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 911/1845] 10.222.0.23 () {60 vars in 1152 bytes} [Thu May 19 12:18:36 2022] POST /identity/v3/projects => generated 375 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 12:18:37.126258 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf25f19a-5cf0-468f-8b46-d9406640ebfc admin admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:18:37.136814 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf25f19a-5cf0-468f-8b46-d9406640ebfc admin admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:18:37.214267 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6f2f0a-6167-4c27-a6be-748364eefad5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:18:37.215413 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6f2f0a-6167-4c27-a6be-748364eefad5 admin admin] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:18:37.215715 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6f2f0a-6167-4c27-a6be-748364eefad5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:18:37.216032 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6f2f0a-6167-4c27-a6be-748364eefad5 admin admin] PATH_INFO: `/v3/users` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:18:37.216781 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6f2f0a-6167-4c27-a6be-748364eefad5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:18:37.218915 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6f2f0a-6167-4c27-a6be-748364eefad5 admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:18:37.447925 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 935/1846] 10.222.0.23 () {60 vars in 1143 bytes} [Thu May 19 12:18:37 2022] POST /identity/v3/users => generated 288 bytes in 322 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 12:18:37.453029 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3efe5e-d798-459c-9ccd-ba932710b09b admin admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:18:37.470471 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3efe5e-d798-459c-9ccd-ba932710b09b admin admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:18:37.478729 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa58411c-08df-4af0-b2a0-7e4be3719b00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:18:37.479993 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa58411c-08df-4af0-b2a0-7e4be3719b00 admin admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:18:37.480579 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa58411c-08df-4af0-b2a0-7e4be3719b00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:18:37.481050 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa58411c-08df-4af0-b2a0-7e4be3719b00 admin admin] PATH_INFO: `/v3/roles` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:18:37.482237 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa58411c-08df-4af0-b2a0-7e4be3719b00 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:18:37.490904 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa58411c-08df-4af0-b2a0-7e4be3719b00 admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:18:37.503920 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 912/1847] 10.222.0.23 () {58 vars in 1121 bytes} [Thu May 19 12:18:37 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 19 12:18:37.508807 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6f2f0a-6167-4c27-a6be-748364eefad5 admin admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:18:37.530979 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6f2f0a-6167-4c27-a6be-748364eefad5 admin admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:18:37.540658 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f2a03e-2bca-4dca-ab86-e77f768cdd8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:18:37.541295 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f2a03e-2bca-4dca-ab86-e77f768cdd8d admin admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:18:37.541734 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f2a03e-2bca-4dca-ab86-e77f768cdd8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:18:37.542276 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f2a03e-2bca-4dca-ab86-e77f768cdd8d admin admin] PATH_INFO: `/v3/roles` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:18:37.544582 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f2a03e-2bca-4dca-ab86-e77f768cdd8d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:18:37.548933 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f2a03e-2bca-4dca-ab86-e77f768cdd8d admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:18:37.567584 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 936/1848] 10.222.0.23 () {58 vars in 1121 bytes} [Thu May 19 12:18:37 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 19 12:18:37.572576 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa58411c-08df-4af0-b2a0-7e4be3719b00 admin admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:18:37.592206 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa58411c-08df-4af0-b2a0-7e4be3719b00 admin admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:18:37.597840 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562bed9b-5956-4093-a8a3-a283a0cab57f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:18:37.599277 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562bed9b-5956-4093-a8a3-a283a0cab57f admin admin] REQUEST_METHOD: `PUT` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:18:37.599697 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562bed9b-5956-4093-a8a3-a283a0cab57f admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:18:37.600020 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562bed9b-5956-4093-a8a3-a283a0cab57f admin admin] PATH_INFO: `/v3/projects/d12b0dc76a4f47758123998bee63907d/users/5030c4dd874c432b8430986f9809b7b6/roles/259eb400cdb4439db72d14456ae9cd08` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:18:37.618906 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562bed9b-5956-4093-a8a3-a283a0cab57f admin admin] RBAC: Authorizing `identity:create_grant(project_id=d12b0dc76a4f47758123998bee63907d, user_id=5030c4dd874c432b8430986f9809b7b6, role_id=259eb400cdb4439db72d14456ae9cd08)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:18:37.621327 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562bed9b-5956-4093-a8a3-a283a0cab57f admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:18:37.779218 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 913/1849] 10.222.0.23 () {60 vars in 1482 bytes} [Thu May 19 12:18:37 2022] PUT /identity/v3/projects/d12b0dc76a4f47758123998bee63907d/users/5030c4dd874c432b8430986f9809b7b6/roles/259eb400cdb4439db72d14456ae9cd08 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:18:37.792212 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871552d8-81da-40a1-9491-cdf65214dbcf None None] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:18:37.792620 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871552d8-81da-40a1-9491-cdf65214dbcf None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:18:37.793007 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871552d8-81da-40a1-9491-cdf65214dbcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:18:37.838695 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.auth.core [None req-871552d8-81da-40a1-9491-cdf65214dbcf None None] MFA Rules not processed for user `5030c4dd874c432b8430986f9809b7b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:18:37.840606 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-871552d8-81da-40a1-9491-cdf65214dbcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:18:38.031834 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 937/1850] 10.222.0.23 () {58 vars in 957 bytes} [Thu May 19 12:18:37 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:18:38.035257 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562bed9b-5956-4093-a8a3-a283a0cab57f admin admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:18:38.112922 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871552d8-81da-40a1-9491-cdf65214dbcf None None] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:18:38.114532 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-871552d8-81da-40a1-9491-cdf65214dbcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:18:38.128334 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:18:38.205824 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871552d8-81da-40a1-9491-cdf65214dbcf None None] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:18:38.212041 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8079ac-007f-4ae3-bfd1-58bf68ded755 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8800254c0e784e42a791fb9abc274d79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd70d25bf69314820b8aaecb65576978a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:18:38.213745 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8079ac-007f-4ae3-bfd1-58bf68ded755 service cinder] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:18:38.214294 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8079ac-007f-4ae3-bfd1-58bf68ded755 service cinder] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:18:38.215148 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8079ac-007f-4ae3-bfd1-58bf68ded755 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:18:38.218039 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-6b8079ac-007f-4ae3-bfd1-58bf68ded755 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:18:38.240952 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562bed9b-5956-4093-a8a3-a283a0cab57f admin admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:18:38.299142 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362f6752-884b-4f4d-8f6b-5f9dbb89b09a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:18:38.301259 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362f6752-884b-4f4d-8f6b-5f9dbb89b09a admin admin] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:18:38.301768 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362f6752-884b-4f4d-8f6b-5f9dbb89b09a admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:18:38.302264 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362f6752-884b-4f4d-8f6b-5f9dbb89b09a admin admin] PATH_INFO: `/v3/projects` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:18:38.305030 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-362f6752-884b-4f4d-8f6b-5f9dbb89b09a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:18:38.313980 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-362f6752-884b-4f4d-8f6b-5f9dbb89b09a admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:18:38.402085 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8079ac-007f-4ae3-bfd1-58bf68ded755 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:18:38.407151 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8079ac-007f-4ae3-bfd1-58bf68ded755 service cinder] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:18:38.433141 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 938/1851] 10.222.0.23 () {60 vars in 1365 bytes} [Thu May 19 12:18:38 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:18:38.557346 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 914/1852] 10.222.0.23 () {60 vars in 1152 bytes} [Thu May 19 12:18:38 2022] POST /identity/v3/projects => generated 375 bytes in 519 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 12:18:38.559946 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8079ac-007f-4ae3-bfd1-58bf68ded755 service cinder] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:18:38.574665 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8079ac-007f-4ae3-bfd1-58bf68ded755 service cinder] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:18:38.827635 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412c61aa-92d9-42f3-9b80-9dd81b43aa96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:18:38.831454 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412c61aa-92d9-42f3-9b80-9dd81b43aa96 admin admin] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:18:38.831454 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412c61aa-92d9-42f3-9b80-9dd81b43aa96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:18:38.831454 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412c61aa-92d9-42f3-9b80-9dd81b43aa96 admin admin] PATH_INFO: `/v3/users` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:18:38.831454 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412c61aa-92d9-42f3-9b80-9dd81b43aa96 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:18:38.832106 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412c61aa-92d9-42f3-9b80-9dd81b43aa96 admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:18:38.993975 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 939/1853] 10.222.0.23 () {60 vars in 1143 bytes} [Thu May 19 12:18:38 2022] POST /identity/v3/users => generated 294 bytes in 436 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 12:18:39.006192 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362f6752-884b-4f4d-8f6b-5f9dbb89b09a admin admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:18:39.059318 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362f6752-884b-4f4d-8f6b-5f9dbb89b09a admin admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:18:39.071508 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db32ee10-a161-46f7-8b64-04253e882a9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:18:39.073671 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db32ee10-a161-46f7-8b64-04253e882a9e admin admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:18:39.074918 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db32ee10-a161-46f7-8b64-04253e882a9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:18:39.075619 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db32ee10-a161-46f7-8b64-04253e882a9e admin admin] PATH_INFO: `/v3/roles` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:18:39.077273 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db32ee10-a161-46f7-8b64-04253e882a9e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:18:39.080869 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db32ee10-a161-46f7-8b64-04253e882a9e admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:18:39.111556 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 915/1854] 10.222.0.23 () {58 vars in 1121 bytes} [Thu May 19 12:18:39 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 12:18:39.117758 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412c61aa-92d9-42f3-9b80-9dd81b43aa96 admin admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:18:39.140569 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412c61aa-92d9-42f3-9b80-9dd81b43aa96 admin admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:18:39.151817 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e28c8e-692e-4ff6-b57e-072e833fd022 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:18:39.153919 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e28c8e-692e-4ff6-b57e-072e833fd022 admin admin] REQUEST_METHOD: `PUT` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:18:39.154889 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e28c8e-692e-4ff6-b57e-072e833fd022 admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:18:39.155267 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e28c8e-692e-4ff6-b57e-072e833fd022 admin admin] PATH_INFO: `/v3/projects/3cdd6be030344eb3a724ae23d1a72b70/users/ef80da8aa6884ebfa994e921754a4338/roles/9db5671c65d3414ea2682afa2ccf1403` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:18:39.184944 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e28c8e-692e-4ff6-b57e-072e833fd022 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3cdd6be030344eb3a724ae23d1a72b70, user_id=ef80da8aa6884ebfa994e921754a4338, role_id=9db5671c65d3414ea2682afa2ccf1403)` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:18:39.189063 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e28c8e-692e-4ff6-b57e-072e833fd022 admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:18:39.310117 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 940/1855] 10.222.0.23 () {60 vars in 1482 bytes} [Thu May 19 12:18:39 2022] PUT /identity/v3/projects/3cdd6be030344eb3a724ae23d1a72b70/users/ef80da8aa6884ebfa994e921754a4338/roles/9db5671c65d3414ea2682afa2ccf1403 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 12:18:39.317676 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db32ee10-a161-46f7-8b64-04253e882a9e admin admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:18:39.385825 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db32ee10-a161-46f7-8b64-04253e882a9e admin admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:18:39.532110 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18215b86-33b8-44cb-93ab-a9ce161115cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:18:39.533984 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18215b86-33b8-44cb-93ab-a9ce161115cb admin admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:18:39.536403 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18215b86-33b8-44cb-93ab-a9ce161115cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:18:39.537168 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18215b86-33b8-44cb-93ab-a9ce161115cb admin admin] PATH_INFO: `/v3/roles` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:18:39.539415 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18215b86-33b8-44cb-93ab-a9ce161115cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:18:39.543196 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18215b86-33b8-44cb-93ab-a9ce161115cb admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:18:39.561360 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 916/1856] 10.222.0.23 () {58 vars in 1121 bytes} [Thu May 19 12:18:39 2022] GET /identity/v3/roles => generated 1420 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 12:18:39.566747 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e28c8e-692e-4ff6-b57e-072e833fd022 admin admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:18:39.580338 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e28c8e-692e-4ff6-b57e-072e833fd022 admin admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:18:39.587154 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91416d93-8c18-468d-bbb3-0e1a07513821 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:18:39.588217 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91416d93-8c18-468d-bbb3-0e1a07513821 admin admin] REQUEST_METHOD: `PUT` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:18:39.588691 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91416d93-8c18-468d-bbb3-0e1a07513821 admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:18:39.588953 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91416d93-8c18-468d-bbb3-0e1a07513821 admin admin] PATH_INFO: `/v3/projects/3cdd6be030344eb3a724ae23d1a72b70/users/ef80da8aa6884ebfa994e921754a4338/roles/9db5671c65d3414ea2682afa2ccf1403` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:18:39.593229 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91416d93-8c18-468d-bbb3-0e1a07513821 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3cdd6be030344eb3a724ae23d1a72b70, user_id=ef80da8aa6884ebfa994e921754a4338, role_id=9db5671c65d3414ea2682afa2ccf1403)` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:18:39.595456 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91416d93-8c18-468d-bbb3-0e1a07513821 admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:18:39.608825 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 941/1857] 10.222.0.23 () {60 vars in 1482 bytes} [Thu May 19 12:18:39 2022] PUT /identity/v3/projects/3cdd6be030344eb3a724ae23d1a72b70/users/ef80da8aa6884ebfa994e921754a4338/roles/9db5671c65d3414ea2682afa2ccf1403 => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:18:39.620577 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1daef41-3c62-4831-bc5f-d018daeb3a8e None None] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:18:39.620850 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1daef41-3c62-4831-bc5f-d018daeb3a8e None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:18:39.621133 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1daef41-3c62-4831-bc5f-d018daeb3a8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:18:39.671870 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.auth.core [None req-b1daef41-3c62-4831-bc5f-d018daeb3a8e None None] MFA Rules not processed for user `ef80da8aa6884ebfa994e921754a4338`. Rule list: `[]` (Enabled: `True`). {{(pid=80539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:18:39.673821 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-b1daef41-3c62-4831-bc5f-d018daeb3a8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:18:39.701659 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:18:39.808940 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 917/1858] 10.222.0.23 () {58 vars in 957 bytes} [Thu May 19 12:18:39 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:18:39.857677 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91416d93-8c18-468d-bbb3-0e1a07513821 admin admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:18:39.881462 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:18:39.939606 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91416d93-8c18-468d-bbb3-0e1a07513821 admin admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:18:39.943317 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7bf6ab8-514b-46d2-8d7e-7d258bea77a5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8800254c0e784e42a791fb9abc274d79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd70d25bf69314820b8aaecb65576978a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:18:39.944021 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bf6ab8-514b-46d2-8d7e-7d258bea77a5 service cinder] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:18:39.944310 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bf6ab8-514b-46d2-8d7e-7d258bea77a5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:18:39.944619 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bf6ab8-514b-46d2-8d7e-7d258bea77a5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:18:40.022262 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7bf6ab8-514b-46d2-8d7e-7d258bea77a5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:18:40.027135 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7bf6ab8-514b-46d2-8d7e-7d258bea77a5 service cinder] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:18:40.052183 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 942/1859] 10.222.0.23 () {60 vars in 1365 bytes} [Thu May 19 12:18:39 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 12:18:52.417695 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=20 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:18:52.748134 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=43 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:18:59.399237 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1daef41-3c62-4831-bc5f-d018daeb3a8e None None] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:18:59.401591 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-b1daef41-3c62-4831-bc5f-d018daeb3a8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:18:59.456617 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1daef41-3c62-4831-bc5f-d018daeb3a8e None None] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:18:59.461401 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32504514-2e77-467c-adf7-d1088d2e7dac service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8800254c0e784e42a791fb9abc274d79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd70d25bf69314820b8aaecb65576978a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:18:59.462166 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32504514-2e77-467c-adf7-d1088d2e7dac service cinder] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:18:59.462447 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32504514-2e77-467c-adf7-d1088d2e7dac service cinder] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:18:59.462728 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32504514-2e77-467c-adf7-d1088d2e7dac service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:18:59.495048 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32504514-2e77-467c-adf7-d1088d2e7dac service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:18:59.496535 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32504514-2e77-467c-adf7-d1088d2e7dac service cinder] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:18:59.509352 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 918/1860] 10.222.0.23 () {60 vars in 1365 bytes} [Thu May 19 12:18:59 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 12:19:01.212877 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7bf6ab8-514b-46d2-8d7e-7d258bea77a5 service cinder] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:19:01.244924 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7bf6ab8-514b-46d2-8d7e-7d258bea77a5 service cinder] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:19:01.260010 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b570596-57b0-4c56-8944-7123f9e31d5e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8800254c0e784e42a791fb9abc274d79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd70d25bf69314820b8aaecb65576978a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:19:01.262049 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b570596-57b0-4c56-8944-7123f9e31d5e service cinder] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:19:01.262289 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b570596-57b0-4c56-8944-7123f9e31d5e service cinder] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:19:01.263112 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b570596-57b0-4c56-8944-7123f9e31d5e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:19:01.265831 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-4b570596-57b0-4c56-8944-7123f9e31d5e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:19:01.380243 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b570596-57b0-4c56-8944-7123f9e31d5e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:19:01.382009 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b570596-57b0-4c56-8944-7123f9e31d5e service cinder] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:19:01.420117 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 943/1861] 10.222.0.23 () {60 vars in 1365 bytes} [Thu May 19 12:19:01 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 199 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 12:19:08.553632 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32504514-2e77-467c-adf7-d1088d2e7dac service cinder] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:19:08.555943 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-32504514-2e77-467c-adf7-d1088d2e7dac service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:19:08.678571 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32504514-2e77-467c-adf7-d1088d2e7dac service cinder] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:19:08.683931 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590cae29-b5f1-4a6f-99dd-6e0e1b8e5360 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:19:08.684805 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590cae29-b5f1-4a6f-99dd-6e0e1b8e5360 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:19:08.685090 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590cae29-b5f1-4a6f-99dd-6e0e1b8e5360 admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:19:08.685371 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590cae29-b5f1-4a6f-99dd-6e0e1b8e5360 admin admin] PATH_INFO: `/v3/users/7f98aebf55bc4f709b3b6e644077371c` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:19:08.686577 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590cae29-b5f1-4a6f-99dd-6e0e1b8e5360 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7f98aebf55bc4f709b3b6e644077371c)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:19:08.688593 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590cae29-b5f1-4a6f-99dd-6e0e1b8e5360 admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:19:08.918728 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-590cae29-b5f1-4a6f-99dd-6e0e1b8e5360 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7f98aebf55bc4f709b3b6e644077371c'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:19:08.935061 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-590cae29-b5f1-4a6f-99dd-6e0e1b8e5360 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7f98aebf55bc4f709b3b6e644077371c'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:19:08.936112 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-590cae29-b5f1-4a6f-99dd-6e0e1b8e5360 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7f98aebf55bc4f709b3b6e644077371c'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:19:09.079590 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-590cae29-b5f1-4a6f-99dd-6e0e1b8e5360 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7f98aebf55bc4f709b3b6e644077371c'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:19:09.083660 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:19:09.089684 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-590cae29-b5f1-4a6f-99dd-6e0e1b8e5360 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7f98aebf55bc4f709b3b6e644077371c'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:19:09.095763 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 919/1862] 10.222.0.23 () {58 vars in 1223 bytes} [Thu May 19 12:19:08 2022] DELETE /identity/v3/users/7f98aebf55bc4f709b3b6e644077371c => generated 0 bytes in 543 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:19:09.100554 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7ddc8a-0b3d-488b-b826-2810969c697b None None] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:19:09.101797 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7ddc8a-0b3d-488b-b826-2810969c697b None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:19:09.101797 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7ddc8a-0b3d-488b-b826-2810969c697b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:19:09.129183 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.auth.core [None req-cd7ddc8a-0b3d-488b-b826-2810969c697b None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:19:09.130404 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-cd7ddc8a-0b3d-488b-b826-2810969c697b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:19:09.193581 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 944/1863] 10.222.0.23 () {58 vars in 957 bytes} [Thu May 19 12:19:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:19:09.203235 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590cae29-b5f1-4a6f-99dd-6e0e1b8e5360 admin admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:19:09.204683 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-590cae29-b5f1-4a6f-99dd-6e0e1b8e5360 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:19:09.291218 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590cae29-b5f1-4a6f-99dd-6e0e1b8e5360 admin admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:19:09.295134 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1acd9e-419d-47ab-b270-73f2051ab79a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a00b3fa7497456895e5e0bf105c4fc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd70d25bf69314820b8aaecb65576978a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:19:09.296023 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1acd9e-419d-47ab-b270-73f2051ab79a service neutron] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:19:09.296268 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1acd9e-419d-47ab-b270-73f2051ab79a service neutron] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:19:09.296633 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1acd9e-419d-47ab-b270-73f2051ab79a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:19:09.309688 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1acd9e-419d-47ab-b270-73f2051ab79a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:19:09.310897 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1acd9e-419d-47ab-b270-73f2051ab79a service neutron] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:19:09.322483 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 920/1864] 10.222.0.23 () {60 vars in 1365 bytes} [Thu May 19 12:19:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:19:10.029879 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7ddc8a-0b3d-488b-b826-2810969c697b None None] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:19:10.030594 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-cd7ddc8a-0b3d-488b-b826-2810969c697b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:19:10.053416 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7ddc8a-0b3d-488b-b826-2810969c697b None None] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:19:10.058988 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec9c438-9e3c-411e-8fc9-7dc35e0ce61f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:19:10.060112 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec9c438-9e3c-411e-8fc9-7dc35e0ce61f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:19:10.060603 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec9c438-9e3c-411e-8fc9-7dc35e0ce61f admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:19:10.060943 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec9c438-9e3c-411e-8fc9-7dc35e0ce61f admin admin] PATH_INFO: `/v3/projects/4798f6c0018540c38a0d0c6276dccafa` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:19:10.063830 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec9c438-9e3c-411e-8fc9-7dc35e0ce61f admin admin] RBAC: Authorizing `identity:delete_project(project_id=4798f6c0018540c38a0d0c6276dccafa)` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:19:10.066669 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec9c438-9e3c-411e-8fc9-7dc35e0ce61f admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:19:10.194030 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-4ec9c438-9e3c-411e-8fc9-7dc35e0ce61f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4798f6c0018540c38a0d0c6276dccafa', 'request_id': 'req-4ec9c438-9e3c-411e-8fc9-7dc35e0ce61f'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:19:10.202236 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-4ec9c438-9e3c-411e-8fc9-7dc35e0ce61f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4798f6c0018540c38a0d0c6276dccafa', 'request_id': 'req-4ec9c438-9e3c-411e-8fc9-7dc35e0ce61f'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:19:10.208119 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-4ec9c438-9e3c-411e-8fc9-7dc35e0ce61f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4798f6c0018540c38a0d0c6276dccafa', 'request_id': 'req-4ec9c438-9e3c-411e-8fc9-7dc35e0ce61f'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:19:10.231380 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1acd9e-419d-47ab-b270-73f2051ab79a service neutron] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:19:10.232939 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-cb1acd9e-419d-47ab-b270-73f2051ab79a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:19:10.246591 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1acd9e-419d-47ab-b270-73f2051ab79a service neutron] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:19:10.305474 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfed90b-8f84-4c16-a492-fbc683bb78a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:19:10.306427 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfed90b-8f84-4c16-a492-fbc683bb78a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:19:10.306712 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfed90b-8f84-4c16-a492-fbc683bb78a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:19:10.306990 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfed90b-8f84-4c16-a492-fbc683bb78a1 admin admin] PATH_INFO: `/v3/users/facbc625174c435591bdad17e472a4fb` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:19:10.308244 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbfed90b-8f84-4c16-a492-fbc683bb78a1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=facbc625174c435591bdad17e472a4fb)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:19:10.310354 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbfed90b-8f84-4c16-a492-fbc683bb78a1 admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:19:10.376711 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-4ec9c438-9e3c-411e-8fc9-7dc35e0ce61f admin admin] The token cache is being invalidate because project 4798f6c0018540c38a0d0c6276dccafa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80540) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 12:19:10.377159 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-4ec9c438-9e3c-411e-8fc9-7dc35e0ce61f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:19:10.377903 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-4ec9c438-9e3c-411e-8fc9-7dc35e0ce61f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:19:10.379564 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 945/1865] 10.222.0.23 () {58 vars in 1232 bytes} [Thu May 19 12:19:10 2022] DELETE /identity/v3/projects/4798f6c0018540c38a0d0c6276dccafa => generated 0 bytes in 351 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:19:10.384841 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3345c41f-7c31-416c-b38a-fed8a762980c None None] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:19:10.385194 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3345c41f-7c31-416c-b38a-fed8a762980c None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:19:10.385421 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3345c41f-7c31-416c-b38a-fed8a762980c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:19:10.390138 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:19:10.438248 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.auth.core [None req-3345c41f-7c31-416c-b38a-fed8a762980c None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:19:10.439477 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-3345c41f-7c31-416c-b38a-fed8a762980c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:19:10.447732 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 946/1866] 10.222.0.23 () {58 vars in 957 bytes} [Thu May 19 12:19:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:19:10.455863 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1807dbe-77a3-4c8b-ab10-e03be291aaed None None] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:19:10.465513 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1807dbe-77a3-4c8b-ab10-e03be291aaed None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:19:10.465513 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1807dbe-77a3-4c8b-ab10-e03be291aaed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:19:10.474469 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.auth.core [None req-e1807dbe-77a3-4c8b-ab10-e03be291aaed None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:19:10.475731 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-e1807dbe-77a3-4c8b-ab10-e03be291aaed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:19:10.484849 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 947/1867] 10.222.0.23 () {58 vars in 957 bytes} [Thu May 19 12:19:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 12:19:10.489198 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1807dbe-77a3-4c8b-ab10-e03be291aaed None None] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:19:10.500535 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1807dbe-77a3-4c8b-ab10-e03be291aaed None None] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:19:10.505326 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cef06c6-7f8b-48fc-b1ad-6f4a5c2749b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:19:10.505997 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cef06c6-7f8b-48fc-b1ad-6f4a5c2749b2 admin admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:19:10.506345 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cef06c6-7f8b-48fc-b1ad-6f4a5c2749b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:19:10.506591 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cef06c6-7f8b-48fc-b1ad-6f4a5c2749b2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:19:10.507689 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cef06c6-7f8b-48fc-b1ad-6f4a5c2749b2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:19:10.508032 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cef06c6-7f8b-48fc-b1ad-6f4a5c2749b2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:19:10.509657 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cef06c6-7f8b-48fc-b1ad-6f4a5c2749b2 admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:19:10.521493 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 948/1868] 10.222.0.23 () {58 vars in 1152 bytes} [Thu May 19 12:19:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 12:19:10.526364 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cef06c6-7f8b-48fc-b1ad-6f4a5c2749b2 admin admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:19:10.536158 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cef06c6-7f8b-48fc-b1ad-6f4a5c2749b2 admin admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:19:10.542404 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef7c017-2ba5-4d33-82ac-243490f87004 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:19:10.543297 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef7c017-2ba5-4d33-82ac-243490f87004 admin admin] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:19:10.543497 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef7c017-2ba5-4d33-82ac-243490f87004 admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:19:10.543780 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef7c017-2ba5-4d33-82ac-243490f87004 admin admin] PATH_INFO: `/v3/projects` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:19:10.544531 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef7c017-2ba5-4d33-82ac-243490f87004 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:19:10.546527 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef7c017-2ba5-4d33-82ac-243490f87004 admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:19:10.570583 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-dbfed90b-8f84-4c16-a492-fbc683bb78a1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'facbc625174c435591bdad17e472a4fb'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:19:10.585779 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-dbfed90b-8f84-4c16-a492-fbc683bb78a1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'facbc625174c435591bdad17e472a4fb'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:19:10.586541 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-dbfed90b-8f84-4c16-a492-fbc683bb78a1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'facbc625174c435591bdad17e472a4fb'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:19:10.646943 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 949/1869] 10.222.0.23 () {60 vars in 1152 bytes} [Thu May 19 12:19:10 2022] POST /identity/v3/projects => generated 361 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 12:19:10.651241 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef7c017-2ba5-4d33-82ac-243490f87004 admin admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:19:10.678190 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-dbfed90b-8f84-4c16-a492-fbc683bb78a1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'facbc625174c435591bdad17e472a4fb'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:19:10.683128 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:19:10.691019 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-dbfed90b-8f84-4c16-a492-fbc683bb78a1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'facbc625174c435591bdad17e472a4fb'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:19:10.701136 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 921/1870] 10.222.0.23 () {58 vars in 1223 bytes} [Thu May 19 12:19:10 2022] DELETE /identity/v3/users/facbc625174c435591bdad17e472a4fb => generated 0 bytes in 467 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 12:19:10.703150 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385b7b31-ba75-4f4e-9724-4ebdd29f2158 None None] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:19:10.703570 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385b7b31-ba75-4f4e-9724-4ebdd29f2158 None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:19:10.703913 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385b7b31-ba75-4f4e-9724-4ebdd29f2158 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:19:10.724062 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.auth.core [None req-385b7b31-ba75-4f4e-9724-4ebdd29f2158 None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:19:10.725435 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-385b7b31-ba75-4f4e-9724-4ebdd29f2158 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:19:10.733726 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef7c017-2ba5-4d33-82ac-243490f87004 admin admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:19:10.807052 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 922/1871] 10.222.0.23 () {58 vars in 957 bytes} [Thu May 19 12:19:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:19:10.811065 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c562b769-36a0-49ce-9b81-922466fbf274 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:19:10.811555 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c562b769-36a0-49ce-9b81-922466fbf274 admin admin] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:19:10.811900 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c562b769-36a0-49ce-9b81-922466fbf274 admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:19:10.812187 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c562b769-36a0-49ce-9b81-922466fbf274 admin admin] PATH_INFO: `/v3/users` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:19:10.812889 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c562b769-36a0-49ce-9b81-922466fbf274 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:19:10.814971 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c562b769-36a0-49ce-9b81-922466fbf274 admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:19:10.819643 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385b7b31-ba75-4f4e-9724-4ebdd29f2158 None None] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:19:10.821864 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-385b7b31-ba75-4f4e-9724-4ebdd29f2158 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:19:10.890052 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385b7b31-ba75-4f4e-9724-4ebdd29f2158 None None] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:19:10.897436 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bcdf80-5587-48a9-acf6-f714143eeaf3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a00b3fa7497456895e5e0bf105c4fc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd70d25bf69314820b8aaecb65576978a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:19:10.899342 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bcdf80-5587-48a9-acf6-f714143eeaf3 service neutron] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:19:10.899695 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bcdf80-5587-48a9-acf6-f714143eeaf3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:19:10.900007 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bcdf80-5587-48a9-acf6-f714143eeaf3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:19:10.927365 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2bcdf80-5587-48a9-acf6-f714143eeaf3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:19:10.929871 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2bcdf80-5587-48a9-acf6-f714143eeaf3 service neutron] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:19:10.941086 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 923/1872] 10.222.0.23 () {60 vars in 1365 bytes} [Thu May 19 12:19:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:19:10.942297 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bcdf80-5587-48a9-acf6-f714143eeaf3 service neutron] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:19:10.943932 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-e2bcdf80-5587-48a9-acf6-f714143eeaf3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:19:10.962061 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bcdf80-5587-48a9-acf6-f714143eeaf3 service neutron] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:19:10.967081 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efcc8bee-0dd5-410f-9b86-1b9c31e6010d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:19:10.968030 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcc8bee-0dd5-410f-9b86-1b9c31e6010d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:19:10.968320 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcc8bee-0dd5-410f-9b86-1b9c31e6010d admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:19:10.968603 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcc8bee-0dd5-410f-9b86-1b9c31e6010d admin admin] PATH_INFO: `/v3/users/57ae03648f2b45588ff418f7daef3120` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:19:10.969890 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efcc8bee-0dd5-410f-9b86-1b9c31e6010d admin admin] RBAC: Authorizing `identity:delete_user(user_id=57ae03648f2b45588ff418f7daef3120)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:19:10.971877 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efcc8bee-0dd5-410f-9b86-1b9c31e6010d admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:19:10.995495 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 950/1873] 10.222.0.23 () {60 vars in 1143 bytes} [Thu May 19 12:19:10 2022] POST /identity/v3/users => generated 281 bytes in 343 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 12:19:10.999771 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c562b769-36a0-49ce-9b81-922466fbf274 admin admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:19:11.001751 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-c562b769-36a0-49ce-9b81-922466fbf274 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:19:11.016979 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c562b769-36a0-49ce-9b81-922466fbf274 admin admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:19:11.026943 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3713be6c-2d70-426c-9152-78a25a7be79e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:19:11.027951 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3713be6c-2d70-426c-9152-78a25a7be79e admin admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:19:11.028305 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3713be6c-2d70-426c-9152-78a25a7be79e admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:19:11.030847 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3713be6c-2d70-426c-9152-78a25a7be79e admin admin] PATH_INFO: `/v3/roles` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:19:11.030847 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3713be6c-2d70-426c-9152-78a25a7be79e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:19:11.032090 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3713be6c-2d70-426c-9152-78a25a7be79e admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:19:11.044516 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 951/1874] 10.222.0.23 () {58 vars in 1121 bytes} [Thu May 19 12:19:10 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 12:19:11.048251 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3713be6c-2d70-426c-9152-78a25a7be79e admin admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:19:11.061299 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3713be6c-2d70-426c-9152-78a25a7be79e admin admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:19:11.064568 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4f209d-672a-4fe0-8a65-a553581b0c59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:19:11.065325 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4f209d-672a-4fe0-8a65-a553581b0c59 admin admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:19:11.065649 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4f209d-672a-4fe0-8a65-a553581b0c59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:19:11.065921 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4f209d-672a-4fe0-8a65-a553581b0c59 admin admin] PATH_INFO: `/v3/roles` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:19:11.066702 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4f209d-672a-4fe0-8a65-a553581b0c59 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:19:11.068362 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4f209d-672a-4fe0-8a65-a553581b0c59 admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:19:11.080363 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 952/1875] 10.222.0.23 () {58 vars in 1121 bytes} [Thu May 19 12:19:11 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 19 12:19:11.084531 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4f209d-672a-4fe0-8a65-a553581b0c59 admin admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:19:11.095173 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4f209d-672a-4fe0-8a65-a553581b0c59 admin admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:19:11.099760 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927d4064-1f15-4d19-9a55-8b04dfe81984 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:19:11.100671 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927d4064-1f15-4d19-9a55-8b04dfe81984 admin admin] REQUEST_METHOD: `PUT` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:19:11.100985 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927d4064-1f15-4d19-9a55-8b04dfe81984 admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:19:11.101263 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927d4064-1f15-4d19-9a55-8b04dfe81984 admin admin] PATH_INFO: `/v3/projects/ffd41da0ef7d4603a9559d65de6c6a21/users/b02a60cc520d467aab55a9c1f03a2423/roles/259eb400cdb4439db72d14456ae9cd08` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:19:11.104094 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:19:11.117550 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927d4064-1f15-4d19-9a55-8b04dfe81984 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ffd41da0ef7d4603a9559d65de6c6a21, user_id=b02a60cc520d467aab55a9c1f03a2423, role_id=259eb400cdb4439db72d14456ae9cd08)` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:19:11.120410 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927d4064-1f15-4d19-9a55-8b04dfe81984 admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:19:11.216858 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-efcc8bee-0dd5-410f-9b86-1b9c31e6010d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '57ae03648f2b45588ff418f7daef3120'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:19:11.235221 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-efcc8bee-0dd5-410f-9b86-1b9c31e6010d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '57ae03648f2b45588ff418f7daef3120'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:19:11.236176 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-efcc8bee-0dd5-410f-9b86-1b9c31e6010d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '57ae03648f2b45588ff418f7daef3120'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:19:11.259544 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 953/1876] 10.222.0.23 () {60 vars in 1482 bytes} [Thu May 19 12:19:11 2022] PUT /identity/v3/projects/ffd41da0ef7d4603a9559d65de6c6a21/users/b02a60cc520d467aab55a9c1f03a2423/roles/259eb400cdb4439db72d14456ae9cd08 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:19:11.269759 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b04f04a-840f-4dc4-8257-d569e31c902c None None] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:19:11.270054 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b04f04a-840f-4dc4-8257-d569e31c902c None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:19:11.270336 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b04f04a-840f-4dc4-8257-d569e31c902c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:19:11.313138 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.auth.core [None req-0b04f04a-840f-4dc4-8257-d569e31c902c None None] MFA Rules not processed for user `b02a60cc520d467aab55a9c1f03a2423`. Rule list: `[]` (Enabled: `True`). {{(pid=80540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:19:11.325392 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-0b04f04a-840f-4dc4-8257-d569e31c902c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:19:11.358260 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-efcc8bee-0dd5-410f-9b86-1b9c31e6010d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '57ae03648f2b45588ff418f7daef3120'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:19:11.358260 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-efcc8bee-0dd5-410f-9b86-1b9c31e6010d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '57ae03648f2b45588ff418f7daef3120'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:19:11.364846 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 924/1877] 10.222.0.23 () {58 vars in 1223 bytes} [Thu May 19 12:19:10 2022] DELETE /identity/v3/users/57ae03648f2b45588ff418f7daef3120 => generated 0 bytes in 423 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:19:11.370077 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3b9901-22d6-4546-a6d1-617de20c12fc None None] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:19:11.370449 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3b9901-22d6-4546-a6d1-617de20c12fc None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:19:11.370960 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3b9901-22d6-4546-a6d1-617de20c12fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:19:11.384065 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:19:11.390685 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.auth.core [None req-4e3b9901-22d6-4546-a6d1-617de20c12fc None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:19:11.392044 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-4e3b9901-22d6-4546-a6d1-617de20c12fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:19:11.400177 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 954/1878] 10.222.0.23 () {58 vars in 957 bytes} [Thu May 19 12:19:11 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:19:11.436210 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b04f04a-840f-4dc4-8257-d569e31c902c None None] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:19:11.437740 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-0b04f04a-840f-4dc4-8257-d569e31c902c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:19:11.465472 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 925/1879] 10.222.0.23 () {58 vars in 957 bytes} [Thu May 19 12:19:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 12:19:11.478339 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3b9901-22d6-4546-a6d1-617de20c12fc None None] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:19:11.484164 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-4e3b9901-22d6-4546-a6d1-617de20c12fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:19:11.512537 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b04f04a-840f-4dc4-8257-d569e31c902c None None] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:19:11.556232 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3b9901-22d6-4546-a6d1-617de20c12fc None None] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:19:11.560375 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e990458-bc7a-4954-8d34-c14e3a74215c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a00b3fa7497456895e5e0bf105c4fc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd70d25bf69314820b8aaecb65576978a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:19:11.561046 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e990458-bc7a-4954-8d34-c14e3a74215c service neutron] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:19:11.561894 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e990458-bc7a-4954-8d34-c14e3a74215c service neutron] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:19:11.562184 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e990458-bc7a-4954-8d34-c14e3a74215c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:19:11.573457 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883e29d8-12b1-445e-a4e6-10b036898fd6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8800254c0e784e42a791fb9abc274d79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd70d25bf69314820b8aaecb65576978a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:19:11.573986 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e990458-bc7a-4954-8d34-c14e3a74215c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:19:11.574142 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883e29d8-12b1-445e-a4e6-10b036898fd6 service cinder] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:19:11.574433 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883e29d8-12b1-445e-a4e6-10b036898fd6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:19:11.574715 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883e29d8-12b1-445e-a4e6-10b036898fd6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:19:11.575813 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e990458-bc7a-4954-8d34-c14e3a74215c service neutron] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:19:11.588355 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 926/1880] 10.222.0.23 () {60 vars in 1365 bytes} [Thu May 19 12:19:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:19:11.589071 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e990458-bc7a-4954-8d34-c14e3a74215c service neutron] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:19:11.590500 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-5e990458-bc7a-4954-8d34-c14e3a74215c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:19:11.606864 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e990458-bc7a-4954-8d34-c14e3a74215c service neutron] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:19:11.612149 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc2d5ea-4224-498b-8371-57dfaa36e318 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:19:11.613052 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc2d5ea-4224-498b-8371-57dfaa36e318 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:19:11.613338 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc2d5ea-4224-498b-8371-57dfaa36e318 admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:19:11.613621 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc2d5ea-4224-498b-8371-57dfaa36e318 admin admin] PATH_INFO: `/v3/projects/0d86579d965f4b58b343e87d2e4960f9` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:19:11.615129 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc2d5ea-4224-498b-8371-57dfaa36e318 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0d86579d965f4b58b343e87d2e4960f9)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:19:11.617815 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc2d5ea-4224-498b-8371-57dfaa36e318 admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:19:11.634382 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:19:11.653920 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883e29d8-12b1-445e-a4e6-10b036898fd6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:19:11.655327 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883e29d8-12b1-445e-a4e6-10b036898fd6 service cinder] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:19:11.666274 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 955/1881] 10.222.0.23 () {60 vars in 1365 bytes} [Thu May 19 12:19:11 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 231 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 12:19:11.681497 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883e29d8-12b1-445e-a4e6-10b036898fd6 service cinder] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:19:11.683212 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-883e29d8-12b1-445e-a4e6-10b036898fd6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:19:11.697923 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883e29d8-12b1-445e-a4e6-10b036898fd6 service cinder] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:19:11.702698 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca10465-1fa3-4d4f-852b-83c96b19a4f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:19:11.703589 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca10465-1fa3-4d4f-852b-83c96b19a4f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:19:11.703905 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca10465-1fa3-4d4f-852b-83c96b19a4f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:19:11.704201 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca10465-1fa3-4d4f-852b-83c96b19a4f8 admin admin] PATH_INFO: `/v3/users/b02a60cc520d467aab55a9c1f03a2423` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:19:11.708218 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca10465-1fa3-4d4f-852b-83c96b19a4f8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b02a60cc520d467aab55a9c1f03a2423)` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:19:11.708218 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca10465-1fa3-4d4f-852b-83c96b19a4f8 admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:19:11.738097 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-7fc2d5ea-4224-498b-8371-57dfaa36e318 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0d86579d965f4b58b343e87d2e4960f9', 'request_id': 'req-7fc2d5ea-4224-498b-8371-57dfaa36e318'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:19:11.744926 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-7fc2d5ea-4224-498b-8371-57dfaa36e318 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0d86579d965f4b58b343e87d2e4960f9', 'request_id': 'req-7fc2d5ea-4224-498b-8371-57dfaa36e318'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:19:11.841339 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-7fc2d5ea-4224-498b-8371-57dfaa36e318 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0d86579d965f4b58b343e87d2e4960f9', 'request_id': 'req-7fc2d5ea-4224-498b-8371-57dfaa36e318'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:19:11.854540 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-7fc2d5ea-4224-498b-8371-57dfaa36e318 admin admin] The token cache is being invalidate because project 0d86579d965f4b58b343e87d2e4960f9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80539) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 12:19:11.855050 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-7fc2d5ea-4224-498b-8371-57dfaa36e318 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:19:11.855751 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-7fc2d5ea-4224-498b-8371-57dfaa36e318 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:19:11.860471 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 927/1882] 10.222.0.23 () {58 vars in 1232 bytes} [Thu May 19 12:19:11 2022] DELETE /identity/v3/projects/0d86579d965f4b58b343e87d2e4960f9 => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:19:11.861384 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc2d5ea-4224-498b-8371-57dfaa36e318 admin admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:19:11.873022 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-7fc2d5ea-4224-498b-8371-57dfaa36e318 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:19:11.970480 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-7ca10465-1fa3-4d4f-852b-83c96b19a4f8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b02a60cc520d467aab55a9c1f03a2423'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:19:11.973621 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc2d5ea-4224-498b-8371-57dfaa36e318 admin admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:19:11.979676 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064201db-459a-441d-ab9b-c2b4c87180d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:19:11.980751 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064201db-459a-441d-ab9b-c2b4c87180d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:19:11.981172 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064201db-459a-441d-ab9b-c2b4c87180d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:19:11.981552 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064201db-459a-441d-ab9b-c2b4c87180d3 admin admin] PATH_INFO: `/v3/users/4f7e7b637163431689dc60301661d398` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:19:11.983740 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-064201db-459a-441d-ab9b-c2b4c87180d3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4f7e7b637163431689dc60301661d398)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:19:11.986573 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-7ca10465-1fa3-4d4f-852b-83c96b19a4f8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b02a60cc520d467aab55a9c1f03a2423'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:19:11.986745 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-064201db-459a-441d-ab9b-c2b4c87180d3 admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:19:11.987590 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-7ca10465-1fa3-4d4f-852b-83c96b19a4f8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b02a60cc520d467aab55a9c1f03a2423'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:19:12.098471 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-7ca10465-1fa3-4d4f-852b-83c96b19a4f8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b02a60cc520d467aab55a9c1f03a2423'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:19:12.104724 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-7ca10465-1fa3-4d4f-852b-83c96b19a4f8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b02a60cc520d467aab55a9c1f03a2423'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:19:12.110932 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 956/1883] 10.222.0.23 () {58 vars in 1223 bytes} [Thu May 19 12:19:11 2022] DELETE /identity/v3/users/b02a60cc520d467aab55a9c1f03a2423 => generated 0 bytes in 430 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:19:12.115518 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ed4812-9550-44d0-9e37-c00f47dc4a2e None None] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:19:12.115937 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ed4812-9550-44d0-9e37-c00f47dc4a2e None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:19:12.116226 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ed4812-9550-44d0-9e37-c00f47dc4a2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:19:12.135233 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.auth.core [None req-97ed4812-9550-44d0-9e37-c00f47dc4a2e None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:19:12.136494 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-97ed4812-9550-44d0-9e37-c00f47dc4a2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:19:12.203093 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 957/1884] 10.222.0.23 () {58 vars in 957 bytes} [Thu May 19 12:19:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 12:19:12.203995 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ed4812-9550-44d0-9e37-c00f47dc4a2e None None] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:19:12.205275 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-97ed4812-9550-44d0-9e37-c00f47dc4a2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:19:12.218077 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ed4812-9550-44d0-9e37-c00f47dc4a2e None None] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:19:12.223652 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe351f65-a224-453f-bc51-4021a1e9546c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:19:12.224537 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe351f65-a224-453f-bc51-4021a1e9546c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:19:12.224822 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe351f65-a224-453f-bc51-4021a1e9546c admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:19:12.225148 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe351f65-a224-453f-bc51-4021a1e9546c admin admin] PATH_INFO: `/v3/projects/1c781d06772d4a11b11d7bacab2cdcb8` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:19:12.226489 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe351f65-a224-453f-bc51-4021a1e9546c admin admin] RBAC: Authorizing `identity:delete_project(project_id=1c781d06772d4a11b11d7bacab2cdcb8)` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:19:12.227276 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-064201db-459a-441d-ab9b-c2b4c87180d3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4f7e7b637163431689dc60301661d398'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:19:12.229348 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe351f65-a224-453f-bc51-4021a1e9546c admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:19:12.241805 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-064201db-459a-441d-ab9b-c2b4c87180d3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4f7e7b637163431689dc60301661d398'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:19:12.243134 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-064201db-459a-441d-ab9b-c2b4c87180d3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4f7e7b637163431689dc60301661d398'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:19:12.363886 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-064201db-459a-441d-ab9b-c2b4c87180d3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4f7e7b637163431689dc60301661d398'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:19:12.373767 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-064201db-459a-441d-ab9b-c2b4c87180d3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4f7e7b637163431689dc60301661d398'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:19:12.383176 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:19:12.385057 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 928/1885] 10.222.0.23 () {58 vars in 1223 bytes} [Thu May 19 12:19:11 2022] DELETE /identity/v3/users/4f7e7b637163431689dc60301661d398 => generated 0 bytes in 524 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:19:12.386035 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064201db-459a-441d-ab9b-c2b4c87180d3 admin admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:19:12.387718 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-064201db-459a-441d-ab9b-c2b4c87180d3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:19:12.408001 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-fe351f65-a224-453f-bc51-4021a1e9546c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1c781d06772d4a11b11d7bacab2cdcb8', 'request_id': 'req-fe351f65-a224-453f-bc51-4021a1e9546c'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:19:12.409423 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:19:12.416961 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-fe351f65-a224-453f-bc51-4021a1e9546c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1c781d06772d4a11b11d7bacab2cdcb8', 'request_id': 'req-fe351f65-a224-453f-bc51-4021a1e9546c'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:19:12.426904 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-fe351f65-a224-453f-bc51-4021a1e9546c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1c781d06772d4a11b11d7bacab2cdcb8', 'request_id': 'req-fe351f65-a224-453f-bc51-4021a1e9546c'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:19:12.458740 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064201db-459a-441d-ab9b-c2b4c87180d3 admin admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:19:12.463226 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ccd376a-68b1-4709-9fdd-055c0e6a7212 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a00b3fa7497456895e5e0bf105c4fc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd70d25bf69314820b8aaecb65576978a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:19:12.463930 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccd376a-68b1-4709-9fdd-055c0e6a7212 service neutron] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:19:12.464219 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccd376a-68b1-4709-9fdd-055c0e6a7212 service neutron] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:19:12.464981 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccd376a-68b1-4709-9fdd-055c0e6a7212 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:19:12.466939 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-0ccd376a-68b1-4709-9fdd-055c0e6a7212 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:19:12.516218 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-fe351f65-a224-453f-bc51-4021a1e9546c admin admin] The token cache is being invalidate because project 1c781d06772d4a11b11d7bacab2cdcb8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80540) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 12:19:12.516875 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-fe351f65-a224-453f-bc51-4021a1e9546c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:19:12.517797 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-fe351f65-a224-453f-bc51-4021a1e9546c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:19:12.519916 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 958/1886] 10.222.0.23 () {58 vars in 1232 bytes} [Thu May 19 12:19:12 2022] DELETE /identity/v3/projects/1c781d06772d4a11b11d7bacab2cdcb8 => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:19:12.524553 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe351f65-a224-453f-bc51-4021a1e9546c admin admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:19:12.527518 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-fe351f65-a224-453f-bc51-4021a1e9546c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:19:12.546008 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ccd376a-68b1-4709-9fdd-055c0e6a7212 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:19:12.547851 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ccd376a-68b1-4709-9fdd-055c0e6a7212 service neutron] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:19:12.560069 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 929/1887] 10.222.0.23 () {60 vars in 1365 bytes} [Thu May 19 12:19:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:19:12.623321 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe351f65-a224-453f-bc51-4021a1e9546c admin admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:19:12.629181 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e7783d-2513-4bad-be73-ba9e0f109446 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:19:12.630641 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e7783d-2513-4bad-be73-ba9e0f109446 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:19:12.630641 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e7783d-2513-4bad-be73-ba9e0f109446 admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:19:12.630955 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e7783d-2513-4bad-be73-ba9e0f109446 admin admin] PATH_INFO: `/v3/users/c4ca456011124b7aacca438fc74feee9` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:19:12.633036 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e7783d-2513-4bad-be73-ba9e0f109446 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c4ca456011124b7aacca438fc74feee9)` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:19:12.635026 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e7783d-2513-4bad-be73-ba9e0f109446 admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:19:13.020263 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-48e7783d-2513-4bad-be73-ba9e0f109446 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c4ca456011124b7aacca438fc74feee9'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:19:13.034196 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-48e7783d-2513-4bad-be73-ba9e0f109446 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c4ca456011124b7aacca438fc74feee9'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:19:13.034917 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-48e7783d-2513-4bad-be73-ba9e0f109446 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c4ca456011124b7aacca438fc74feee9'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:19:13.126581 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ccd376a-68b1-4709-9fdd-055c0e6a7212 service neutron] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:19:13.128334 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-0ccd376a-68b1-4709-9fdd-055c0e6a7212 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:19:13.151972 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ccd376a-68b1-4709-9fdd-055c0e6a7212 service neutron] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:19:13.164770 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08201d8b-9b1d-4eca-956a-bfe4c5132751 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:19:13.164770 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08201d8b-9b1d-4eca-956a-bfe4c5132751 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:19:13.164770 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08201d8b-9b1d-4eca-956a-bfe4c5132751 admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:19:13.164770 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08201d8b-9b1d-4eca-956a-bfe4c5132751 admin admin] PATH_INFO: `/v3/projects/633bf351dd7c494e9bbabd2704f0e517` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:19:13.165390 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08201d8b-9b1d-4eca-956a-bfe4c5132751 admin admin] RBAC: Authorizing `identity:delete_project(project_id=633bf351dd7c494e9bbabd2704f0e517)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:19:13.165390 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08201d8b-9b1d-4eca-956a-bfe4c5132751 admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:19:13.184339 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-48e7783d-2513-4bad-be73-ba9e0f109446 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c4ca456011124b7aacca438fc74feee9'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:19:13.196404 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-48e7783d-2513-4bad-be73-ba9e0f109446 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c4ca456011124b7aacca438fc74feee9'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:19:13.203582 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 959/1888] 10.222.0.23 () {58 vars in 1223 bytes} [Thu May 19 12:19:12 2022] DELETE /identity/v3/users/c4ca456011124b7aacca438fc74feee9 => generated 0 bytes in 680 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:19:13.208240 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e7783d-2513-4bad-be73-ba9e0f109446 admin admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:19:13.209970 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-48e7783d-2513-4bad-be73-ba9e0f109446 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:19:13.302967 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e7783d-2513-4bad-be73-ba9e0f109446 admin admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:19:13.319468 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-08201d8b-9b1d-4eca-956a-bfe4c5132751 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '633bf351dd7c494e9bbabd2704f0e517', 'request_id': 'req-08201d8b-9b1d-4eca-956a-bfe4c5132751'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:19:13.328722 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-08201d8b-9b1d-4eca-956a-bfe4c5132751 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '633bf351dd7c494e9bbabd2704f0e517', 'request_id': 'req-08201d8b-9b1d-4eca-956a-bfe4c5132751'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:19:13.379404 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23df27d6-e216-4d4f-b76d-3d8eee965478 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:19:13.380391 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23df27d6-e216-4d4f-b76d-3d8eee965478 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:19:13.380862 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23df27d6-e216-4d4f-b76d-3d8eee965478 admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:19:13.380970 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23df27d6-e216-4d4f-b76d-3d8eee965478 admin admin] PATH_INFO: `/v3/projects/ffd41da0ef7d4603a9559d65de6c6a21` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:19:13.382146 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23df27d6-e216-4d4f-b76d-3d8eee965478 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ffd41da0ef7d4603a9559d65de6c6a21)` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:19:13.385145 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23df27d6-e216-4d4f-b76d-3d8eee965478 admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:19:13.410030 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-08201d8b-9b1d-4eca-956a-bfe4c5132751 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '633bf351dd7c494e9bbabd2704f0e517', 'request_id': 'req-08201d8b-9b1d-4eca-956a-bfe4c5132751'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:19:13.423103 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:19:13.424727 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-08201d8b-9b1d-4eca-956a-bfe4c5132751 admin admin] The token cache is being invalidate because project 633bf351dd7c494e9bbabd2704f0e517 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80539) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 12:19:13.425236 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-08201d8b-9b1d-4eca-956a-bfe4c5132751 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:19:13.426003 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-08201d8b-9b1d-4eca-956a-bfe4c5132751 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:19:13.431007 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 930/1889] 10.222.0.23 () {58 vars in 1232 bytes} [Thu May 19 12:19:13 2022] DELETE /identity/v3/projects/633bf351dd7c494e9bbabd2704f0e517 => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 12:19:13.433694 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d411f3f-8bea-4db9-954d-b957f79bebf2 None None] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:19:13.433694 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d411f3f-8bea-4db9-954d-b957f79bebf2 None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:19:13.434204 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d411f3f-8bea-4db9-954d-b957f79bebf2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:19:13.462829 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.auth.core [None req-3d411f3f-8bea-4db9-954d-b957f79bebf2 None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:19:13.463159 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-3d411f3f-8bea-4db9-954d-b957f79bebf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:19:13.475324 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 931/1890] 10.222.0.23 () {58 vars in 957 bytes} [Thu May 19 12:19:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:19:13.497339 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c382b6af-a0d2-4423-863c-dabdbfaa13bf None None] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:19:13.497339 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c382b6af-a0d2-4423-863c-dabdbfaa13bf None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:19:13.497339 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c382b6af-a0d2-4423-863c-dabdbfaa13bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:19:13.504822 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.auth.core [None req-c382b6af-a0d2-4423-863c-dabdbfaa13bf None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:19:13.506021 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-c382b6af-a0d2-4423-863c-dabdbfaa13bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:19:13.517786 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 932/1891] 10.222.0.23 () {58 vars in 957 bytes} [Thu May 19 12:19:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:19:13.521242 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:19:13.525022 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c382b6af-a0d2-4423-863c-dabdbfaa13bf None None] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:19:13.532182 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-23df27d6-e216-4d4f-b76d-3d8eee965478 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ffd41da0ef7d4603a9559d65de6c6a21', 'request_id': 'req-23df27d6-e216-4d4f-b76d-3d8eee965478'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:19:13.535386 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c382b6af-a0d2-4423-863c-dabdbfaa13bf None None] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:19:13.540166 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-23df27d6-e216-4d4f-b76d-3d8eee965478 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ffd41da0ef7d4603a9559d65de6c6a21', 'request_id': 'req-23df27d6-e216-4d4f-b76d-3d8eee965478'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:19:13.547684 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-23df27d6-e216-4d4f-b76d-3d8eee965478 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ffd41da0ef7d4603a9559d65de6c6a21', 'request_id': 'req-23df27d6-e216-4d4f-b76d-3d8eee965478'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:19:13.599111 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392fa081-0e0a-4ecd-9e9a-a04499f22159 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:19:13.599893 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392fa081-0e0a-4ecd-9e9a-a04499f22159 admin admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:19:13.600274 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392fa081-0e0a-4ecd-9e9a-a04499f22159 admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:19:13.600591 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392fa081-0e0a-4ecd-9e9a-a04499f22159 admin admin] PATH_INFO: `/v3/domains` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:19:13.601803 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392fa081-0e0a-4ecd-9e9a-a04499f22159 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:19:13.601803 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392fa081-0e0a-4ecd-9e9a-a04499f22159 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:19:13.603871 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392fa081-0e0a-4ecd-9e9a-a04499f22159 admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:19:13.615845 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 933/1892] 10.222.0.23 () {58 vars in 1152 bytes} [Thu May 19 12:19:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:19:13.621068 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392fa081-0e0a-4ecd-9e9a-a04499f22159 admin admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:19:13.630922 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392fa081-0e0a-4ecd-9e9a-a04499f22159 admin admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:19:13.637888 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d4ad59-ebe0-4f54-8273-952f9dc3ce45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:19:13.638169 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d4ad59-ebe0-4f54-8273-952f9dc3ce45 admin admin] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:19:13.638298 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d4ad59-ebe0-4f54-8273-952f9dc3ce45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:19:13.638513 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d4ad59-ebe0-4f54-8273-952f9dc3ce45 admin admin] PATH_INFO: `/v3/projects` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:19:13.639208 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d4ad59-ebe0-4f54-8273-952f9dc3ce45 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:19:13.641145 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d4ad59-ebe0-4f54-8273-952f9dc3ce45 admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:19:13.650471 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-23df27d6-e216-4d4f-b76d-3d8eee965478 admin admin] The token cache is being invalidate because project ffd41da0ef7d4603a9559d65de6c6a21 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80540) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 12:19:13.656448 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-23df27d6-e216-4d4f-b76d-3d8eee965478 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:19:13.656448 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-23df27d6-e216-4d4f-b76d-3d8eee965478 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:19:13.656448 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 960/1893] 10.222.0.23 () {58 vars in 1232 bytes} [Thu May 19 12:19:13 2022] DELETE /identity/v3/projects/ffd41da0ef7d4603a9559d65de6c6a21 => generated 0 bytes in 445 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:19:13.659067 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e8d905-a52c-48c9-be2f-b2e3d4fa8aac None None] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:19:13.659360 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e8d905-a52c-48c9-be2f-b2e3d4fa8aac None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:19:13.659660 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e8d905-a52c-48c9-be2f-b2e3d4fa8aac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:19:13.687355 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.auth.core [None req-28e8d905-a52c-48c9-be2f-b2e3d4fa8aac None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:19:13.690915 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-28e8d905-a52c-48c9-be2f-b2e3d4fa8aac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:19:13.700585 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 961/1894] 10.222.0.23 () {58 vars in 957 bytes} [Thu May 19 12:19:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 12:19:13.701484 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e8d905-a52c-48c9-be2f-b2e3d4fa8aac None None] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:19:13.702845 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-28e8d905-a52c-48c9-be2f-b2e3d4fa8aac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:19:13.716575 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e8d905-a52c-48c9-be2f-b2e3d4fa8aac None None] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:19:13.722370 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cef0ef3-88db-462a-abf7-1b92c7c5069a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:19:13.723255 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cef0ef3-88db-462a-abf7-1b92c7c5069a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:19:13.723541 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cef0ef3-88db-462a-abf7-1b92c7c5069a admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:19:13.723849 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cef0ef3-88db-462a-abf7-1b92c7c5069a admin admin] PATH_INFO: `/v3/projects/3f2a1b6c9bd74b8c825d17d4362c01e9` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:19:13.725067 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cef0ef3-88db-462a-abf7-1b92c7c5069a admin admin] RBAC: Authorizing `identity:delete_project(project_id=3f2a1b6c9bd74b8c825d17d4362c01e9)` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:19:13.727127 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cef0ef3-88db-462a-abf7-1b92c7c5069a admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:19:13.757920 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 934/1895] 10.222.0.23 () {60 vars in 1152 bytes} [Thu May 19 12:19:13 2022] POST /identity/v3/projects => generated 365 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 12:19:13.760199 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9375f4-0631-4d8e-b34c-b65d8ed916ba None None] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:19:13.760701 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9375f4-0631-4d8e-b34c-b65d8ed916ba None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:19:13.761162 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9375f4-0631-4d8e-b34c-b65d8ed916ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:19:13.781238 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.auth.core [None req-0d9375f4-0631-4d8e-b34c-b65d8ed916ba None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:19:13.782469 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-0d9375f4-0631-4d8e-b34c-b65d8ed916ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:19:13.846051 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 935/1896] 10.222.0.23 () {58 vars in 957 bytes} [Thu May 19 12:19:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:19:13.846257 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-4cef0ef3-88db-462a-abf7-1b92c7c5069a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3f2a1b6c9bd74b8c825d17d4362c01e9', 'request_id': 'req-4cef0ef3-88db-462a-abf7-1b92c7c5069a'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:19:13.846914 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9375f4-0631-4d8e-b34c-b65d8ed916ba None None] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:19:13.848242 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-0d9375f4-0631-4d8e-b34c-b65d8ed916ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:19:13.854153 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-4cef0ef3-88db-462a-abf7-1b92c7c5069a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3f2a1b6c9bd74b8c825d17d4362c01e9', 'request_id': 'req-4cef0ef3-88db-462a-abf7-1b92c7c5069a'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:19:13.860020 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-4cef0ef3-88db-462a-abf7-1b92c7c5069a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3f2a1b6c9bd74b8c825d17d4362c01e9', 'request_id': 'req-4cef0ef3-88db-462a-abf7-1b92c7c5069a'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:19:13.861232 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9375f4-0631-4d8e-b34c-b65d8ed916ba None None] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:19:13.917999 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1bdbc25-a8cf-4035-8598-28660ccffd14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:19:13.918677 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bdbc25-a8cf-4035-8598-28660ccffd14 admin admin] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:19:13.918961 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bdbc25-a8cf-4035-8598-28660ccffd14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:19:13.919237 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bdbc25-a8cf-4035-8598-28660ccffd14 admin admin] PATH_INFO: `/v3/users` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:19:13.919959 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1bdbc25-a8cf-4035-8598-28660ccffd14 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:19:13.921884 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1bdbc25-a8cf-4035-8598-28660ccffd14 admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:19:13.968982 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-4cef0ef3-88db-462a-abf7-1b92c7c5069a admin admin] The token cache is being invalidate because project 3f2a1b6c9bd74b8c825d17d4362c01e9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80540) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 12:19:13.969263 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-4cef0ef3-88db-462a-abf7-1b92c7c5069a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:19:13.969923 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-4cef0ef3-88db-462a-abf7-1b92c7c5069a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:19:13.971685 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 962/1897] 10.222.0.23 () {58 vars in 1232 bytes} [Thu May 19 12:19:13 2022] DELETE /identity/v3/projects/3f2a1b6c9bd74b8c825d17d4362c01e9 => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:19:13.972532 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cef0ef3-88db-462a-abf7-1b92c7c5069a admin admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:19:13.975015 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-4cef0ef3-88db-462a-abf7-1b92c7c5069a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:19:13.987319 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cef0ef3-88db-462a-abf7-1b92c7c5069a admin admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:19:13.992848 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab6c4d3-6ac1-4533-beca-6e38cc109253 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:19:13.993481 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab6c4d3-6ac1-4533-beca-6e38cc109253 admin admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:19:13.993772 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab6c4d3-6ac1-4533-beca-6e38cc109253 admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:19:13.994051 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab6c4d3-6ac1-4533-beca-6e38cc109253 admin admin] PATH_INFO: `/v3/domains` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:19:13.994796 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab6c4d3-6ac1-4533-beca-6e38cc109253 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:19:13.995107 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab6c4d3-6ac1-4533-beca-6e38cc109253 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:19:13.996853 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab6c4d3-6ac1-4533-beca-6e38cc109253 admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:19:14.012982 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 963/1898] 10.222.0.23 () {58 vars in 1152 bytes} [Thu May 19 12:19:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:19:14.012982 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab6c4d3-6ac1-4533-beca-6e38cc109253 admin admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:19:14.012982 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-aab6c4d3-6ac1-4533-beca-6e38cc109253 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:19:14.026347 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab6c4d3-6ac1-4533-beca-6e38cc109253 admin admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:19:14.032449 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a714f24-3b3a-455b-b3ea-a68cecef5098 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:19:14.033328 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a714f24-3b3a-455b-b3ea-a68cecef5098 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:19:14.033610 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a714f24-3b3a-455b-b3ea-a68cecef5098 admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:19:14.033925 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a714f24-3b3a-455b-b3ea-a68cecef5098 admin admin] PATH_INFO: `/v3/users/df27066a0d674d7ca3c0b2b22f61f2dd` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:19:14.035322 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a714f24-3b3a-455b-b3ea-a68cecef5098 admin admin] RBAC: Authorizing `identity:delete_user(user_id=df27066a0d674d7ca3c0b2b22f61f2dd)` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:19:14.037298 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a714f24-3b3a-455b-b3ea-a68cecef5098 admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:19:14.106482 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 936/1899] 10.222.0.23 () {60 vars in 1143 bytes} [Thu May 19 12:19:13 2022] POST /identity/v3/users => generated 283 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 12:19:14.106482 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1bdbc25-a8cf-4035-8598-28660ccffd14 admin admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:19:14.115046 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1bdbc25-a8cf-4035-8598-28660ccffd14 admin admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:19:14.120728 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe4e708-8d79-4814-aab4-71fcd0c4283e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:19:14.121560 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe4e708-8d79-4814-aab4-71fcd0c4283e admin admin] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:19:14.121744 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe4e708-8d79-4814-aab4-71fcd0c4283e admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:19:14.122028 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe4e708-8d79-4814-aab4-71fcd0c4283e admin admin] PATH_INFO: `/v3/projects` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:19:14.122733 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe4e708-8d79-4814-aab4-71fcd0c4283e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:19:14.124704 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe4e708-8d79-4814-aab4-71fcd0c4283e admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:19:14.231881 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 937/1900] 10.222.0.23 () {60 vars in 1152 bytes} [Thu May 19 12:19:14 2022] POST /identity/v3/projects => generated 373 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 12:19:14.233124 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe4e708-8d79-4814-aab4-71fcd0c4283e admin admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:19:14.235598 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-3fe4e708-8d79-4814-aab4-71fcd0c4283e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:19:14.241505 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-2a714f24-3b3a-455b-b3ea-a68cecef5098 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'df27066a0d674d7ca3c0b2b22f61f2dd'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:19:14.256690 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-2a714f24-3b3a-455b-b3ea-a68cecef5098 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'df27066a0d674d7ca3c0b2b22f61f2dd'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:19:14.257607 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-2a714f24-3b3a-455b-b3ea-a68cecef5098 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'df27066a0d674d7ca3c0b2b22f61f2dd'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:19:14.314939 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe4e708-8d79-4814-aab4-71fcd0c4283e admin admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:19:14.318649 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f351fb72-58f3-4a3b-b1ce-2ce1b55f5540 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:19:14.319483 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f351fb72-58f3-4a3b-b1ce-2ce1b55f5540 admin admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:19:14.319995 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f351fb72-58f3-4a3b-b1ce-2ce1b55f5540 admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:19:14.321163 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f351fb72-58f3-4a3b-b1ce-2ce1b55f5540 admin admin] PATH_INFO: `/v3/roles` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:19:14.321363 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f351fb72-58f3-4a3b-b1ce-2ce1b55f5540 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:19:14.323159 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f351fb72-58f3-4a3b-b1ce-2ce1b55f5540 admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:19:14.334276 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 938/1901] 10.222.0.23 () {58 vars in 1121 bytes} [Thu May 19 12:19:14 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 12:19:14.335163 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f351fb72-58f3-4a3b-b1ce-2ce1b55f5540 admin admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:19:14.339608 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-2a714f24-3b3a-455b-b3ea-a68cecef5098 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'df27066a0d674d7ca3c0b2b22f61f2dd'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:19:14.347427 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f351fb72-58f3-4a3b-b1ce-2ce1b55f5540 admin admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:19:14.349126 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-2a714f24-3b3a-455b-b3ea-a68cecef5098 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'df27066a0d674d7ca3c0b2b22f61f2dd'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:19:14.354121 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf086835-473c-4f3d-8cf1-582bf41fdc17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:19:14.354931 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf086835-473c-4f3d-8cf1-582bf41fdc17 admin admin] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:19:14.355279 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf086835-473c-4f3d-8cf1-582bf41fdc17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:19:14.355554 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf086835-473c-4f3d-8cf1-582bf41fdc17 admin admin] PATH_INFO: `/v3/users` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:19:14.356422 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf086835-473c-4f3d-8cf1-582bf41fdc17 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:19:14.361278 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 964/1902] 10.222.0.23 () {58 vars in 1223 bytes} [Thu May 19 12:19:14 2022] DELETE /identity/v3/users/df27066a0d674d7ca3c0b2b22f61f2dd => generated 0 bytes in 347 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:19:14.361890 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf086835-473c-4f3d-8cf1-582bf41fdc17 admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:19:14.362014 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a714f24-3b3a-455b-b3ea-a68cecef5098 admin admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:19:14.362379 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-2a714f24-3b3a-455b-b3ea-a68cecef5098 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:19:14.443049 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a714f24-3b3a-455b-b3ea-a68cecef5098 admin admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:19:14.449206 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba95ca8-884c-4610-8dbd-37d2f066309b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:19:14.449988 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba95ca8-884c-4610-8dbd-37d2f066309b admin admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:19:14.450492 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba95ca8-884c-4610-8dbd-37d2f066309b admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:19:14.450808 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba95ca8-884c-4610-8dbd-37d2f066309b admin admin] PATH_INFO: `/v3/roles` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:19:14.451751 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba95ca8-884c-4610-8dbd-37d2f066309b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:19:14.453497 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba95ca8-884c-4610-8dbd-37d2f066309b admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:19:14.467827 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 965/1903] 10.222.0.23 () {58 vars in 1121 bytes} [Thu May 19 12:19:14 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 12:19:14.471604 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba95ca8-884c-4610-8dbd-37d2f066309b admin admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:19:14.482373 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 939/1904] 10.222.0.23 () {60 vars in 1143 bytes} [Thu May 19 12:19:14 2022] POST /identity/v3/users => generated 287 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 12:19:14.486215 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba95ca8-884c-4610-8dbd-37d2f066309b admin admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:19:14.489519 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf086835-473c-4f3d-8cf1-582bf41fdc17 admin admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:19:14.491258 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-bf086835-473c-4f3d-8cf1-582bf41fdc17 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:19:14.492770 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756a492a-7804-4dd7-8c02-0d1d5b18d6aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:19:14.493259 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756a492a-7804-4dd7-8c02-0d1d5b18d6aa admin admin] REQUEST_METHOD: `PUT` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:19:14.493603 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756a492a-7804-4dd7-8c02-0d1d5b18d6aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:19:14.494000 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756a492a-7804-4dd7-8c02-0d1d5b18d6aa admin admin] PATH_INFO: `/v3/projects/3e55f1346ff74acc9aa28661a7f079d5/users/a9166286774546a496bae346f0af332d/roles/259eb400cdb4439db72d14456ae9cd08` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:19:14.505926 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf086835-473c-4f3d-8cf1-582bf41fdc17 admin admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:19:14.512084 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09927039-be61-45d5-a5f1-644a7219571e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:19:14.512889 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09927039-be61-45d5-a5f1-644a7219571e admin admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:19:14.513102 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09927039-be61-45d5-a5f1-644a7219571e admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:19:14.513339 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09927039-be61-45d5-a5f1-644a7219571e admin admin] PATH_INFO: `/v3/roles` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:19:14.514208 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09927039-be61-45d5-a5f1-644a7219571e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:19:14.514717 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-756a492a-7804-4dd7-8c02-0d1d5b18d6aa admin admin] RBAC: Authorizing `identity:create_grant(project_id=3e55f1346ff74acc9aa28661a7f079d5, user_id=a9166286774546a496bae346f0af332d, role_id=259eb400cdb4439db72d14456ae9cd08)` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:19:14.516083 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09927039-be61-45d5-a5f1-644a7219571e admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:19:14.517595 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-756a492a-7804-4dd7-8c02-0d1d5b18d6aa admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:19:14.530105 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 940/1905] 10.222.0.23 () {58 vars in 1121 bytes} [Thu May 19 12:19:14 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 19 12:19:14.534161 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09927039-be61-45d5-a5f1-644a7219571e admin admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:19:14.545772 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09927039-be61-45d5-a5f1-644a7219571e admin admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:19:14.551141 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48898af-2111-40da-8765-b0edf884b6cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:19:14.551772 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48898af-2111-40da-8765-b0edf884b6cb admin admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:19:14.552160 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48898af-2111-40da-8765-b0edf884b6cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:19:14.552558 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48898af-2111-40da-8765-b0edf884b6cb admin admin] PATH_INFO: `/v3/roles` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:19:14.553237 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48898af-2111-40da-8765-b0edf884b6cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:19:14.555097 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48898af-2111-40da-8765-b0edf884b6cb admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:19:14.566418 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 941/1906] 10.222.0.23 () {58 vars in 1121 bytes} [Thu May 19 12:19:14 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 19 12:19:14.572822 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48898af-2111-40da-8765-b0edf884b6cb admin admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:19:14.583240 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48898af-2111-40da-8765-b0edf884b6cb admin admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:19:14.590454 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b775ac8c-2fc5-4f59-8787-f58e39763d07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:19:14.591713 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b775ac8c-2fc5-4f59-8787-f58e39763d07 admin admin] REQUEST_METHOD: `PUT` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:19:14.592016 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b775ac8c-2fc5-4f59-8787-f58e39763d07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:19:14.592354 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b775ac8c-2fc5-4f59-8787-f58e39763d07 admin admin] PATH_INFO: `/v3/projects/9bf39a0b49fc46ab8691bb59fa4901ae/users/2985089453f340e0a9bf62d8fc012031/roles/259eb400cdb4439db72d14456ae9cd08` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:19:14.612680 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b775ac8c-2fc5-4f59-8787-f58e39763d07 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9bf39a0b49fc46ab8691bb59fa4901ae, user_id=2985089453f340e0a9bf62d8fc012031, role_id=259eb400cdb4439db72d14456ae9cd08)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:19:14.615855 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b775ac8c-2fc5-4f59-8787-f58e39763d07 admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:19:14.635683 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 966/1907] 10.222.0.23 () {60 vars in 1482 bytes} [Thu May 19 12:19:14 2022] PUT /identity/v3/projects/3e55f1346ff74acc9aa28661a7f079d5/users/a9166286774546a496bae346f0af332d/roles/259eb400cdb4439db72d14456ae9cd08 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 12:19:14.648714 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdfc9f2-0cf6-45d7-a035-7cfdd9e74a11 None None] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:19:14.650349 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdfc9f2-0cf6-45d7-a035-7cfdd9e74a11 None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:19:14.651026 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdfc9f2-0cf6-45d7-a035-7cfdd9e74a11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:19:14.686358 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.auth.core [None req-8cdfc9f2-0cf6-45d7-a035-7cfdd9e74a11 None None] MFA Rules not processed for user `a9166286774546a496bae346f0af332d`. Rule list: `[]` (Enabled: `True`). {{(pid=80540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:19:14.687395 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-8cdfc9f2-0cf6-45d7-a035-7cfdd9e74a11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:19:14.731532 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 942/1908] 10.222.0.23 () {60 vars in 1482 bytes} [Thu May 19 12:19:14 2022] PUT /identity/v3/projects/9bf39a0b49fc46ab8691bb59fa4901ae/users/2985089453f340e0a9bf62d8fc012031/roles/259eb400cdb4439db72d14456ae9cd08 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 12:19:14.733436 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674593c1-ce66-4303-baf7-6eae30ef56bf None None] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:19:14.733744 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674593c1-ce66-4303-baf7-6eae30ef56bf None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:19:14.734016 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674593c1-ce66-4303-baf7-6eae30ef56bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:19:14.763015 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.auth.core [None req-674593c1-ce66-4303-baf7-6eae30ef56bf None None] MFA Rules not processed for user `2985089453f340e0a9bf62d8fc012031`. Rule list: `[]` (Enabled: `True`). {{(pid=80539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:19:14.764287 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-674593c1-ce66-4303-baf7-6eae30ef56bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:19:14.771672 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 967/1909] 10.222.0.23 () {58 vars in 957 bytes} [Thu May 19 12:19:14 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:19:14.777970 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cdfc9f2-0cf6-45d7-a035-7cfdd9e74a11 None None] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:19:14.794444 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:19:14.847167 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 943/1910] 10.222.0.23 () {58 vars in 957 bytes} [Thu May 19 12:19:14 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:19:14.852681 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674593c1-ce66-4303-baf7-6eae30ef56bf None None] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:19:14.861225 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cdfc9f2-0cf6-45d7-a035-7cfdd9e74a11 None None] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:19:14.884521 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:19:14.941058 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a121d22-d177-41a0-80fb-251bc1210056 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:19:14.941734 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a121d22-d177-41a0-80fb-251bc1210056 admin admin] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:19:14.942023 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a121d22-d177-41a0-80fb-251bc1210056 admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:19:14.942303 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a121d22-d177-41a0-80fb-251bc1210056 admin admin] PATH_INFO: `/v3/projects` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:19:14.943010 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a121d22-d177-41a0-80fb-251bc1210056 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:19:14.956182 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674593c1-ce66-4303-baf7-6eae30ef56bf None None] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:19:14.956375 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a121d22-d177-41a0-80fb-251bc1210056 admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:19:14.975411 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a90101-a9d0-4576-8009-1adefccd959d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:19:14.977290 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a90101-a9d0-4576-8009-1adefccd959d admin admin] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:19:14.977592 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a90101-a9d0-4576-8009-1adefccd959d admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:19:14.978166 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a90101-a9d0-4576-8009-1adefccd959d admin admin] PATH_INFO: `/v3/projects` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:19:14.978972 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a90101-a9d0-4576-8009-1adefccd959d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:19:14.981273 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a90101-a9d0-4576-8009-1adefccd959d admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:19:15.101662 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 968/1911] 10.222.0.23 () {60 vars in 1152 bytes} [Thu May 19 12:19:14 2022] POST /identity/v3/projects => generated 363 bytes in 322 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 12:19:15.101662 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a121d22-d177-41a0-80fb-251bc1210056 admin admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:19:15.103628 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-6a121d22-d177-41a0-80fb-251bc1210056 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:19:15.114570 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 944/1912] 10.222.0.23 () {60 vars in 1152 bytes} [Thu May 19 12:19:14 2022] POST /identity/v3/projects => generated 371 bytes in 260 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 12:19:15.114570 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a90101-a9d0-4576-8009-1adefccd959d admin admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:19:15.179451 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a121d22-d177-41a0-80fb-251bc1210056 admin admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:19:15.184274 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3b3a86-dd7e-4912-904c-b319dc407973 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:19:15.185168 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3b3a86-dd7e-4912-904c-b319dc407973 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:19:15.185458 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a90101-a9d0-4576-8009-1adefccd959d admin admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:19:15.185559 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3b3a86-dd7e-4912-904c-b319dc407973 admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:19:15.185718 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3b3a86-dd7e-4912-904c-b319dc407973 admin admin] PATH_INFO: `/v3/projects/238a310adf454901970f67a73b78c4dc` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:19:15.187004 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3b3a86-dd7e-4912-904c-b319dc407973 admin admin] RBAC: Authorizing `identity:delete_project(project_id=238a310adf454901970f67a73b78c4dc)` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:19:15.189058 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3b3a86-dd7e-4912-904c-b319dc407973 admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:19:15.190297 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6545c1a4-dee8-4af0-97dd-1be139cccec4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:19:15.190996 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6545c1a4-dee8-4af0-97dd-1be139cccec4 admin admin] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:19:15.191278 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6545c1a4-dee8-4af0-97dd-1be139cccec4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:19:15.191568 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6545c1a4-dee8-4af0-97dd-1be139cccec4 admin admin] PATH_INFO: `/v3/users` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:19:15.192581 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6545c1a4-dee8-4af0-97dd-1be139cccec4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:19:15.194408 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6545c1a4-dee8-4af0-97dd-1be139cccec4 admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:19:15.313693 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 945/1913] 10.222.0.23 () {60 vars in 1143 bytes} [Thu May 19 12:19:15 2022] POST /identity/v3/users => generated 288 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 12:19:15.313693 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6545c1a4-dee8-4af0-97dd-1be139cccec4 admin admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:19:15.323949 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-5b3b3a86-dd7e-4912-904c-b319dc407973 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '238a310adf454901970f67a73b78c4dc', 'request_id': 'req-5b3b3a86-dd7e-4912-904c-b319dc407973'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:19:15.324801 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6545c1a4-dee8-4af0-97dd-1be139cccec4 admin admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:19:15.332094 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-5b3b3a86-dd7e-4912-904c-b319dc407973 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '238a310adf454901970f67a73b78c4dc', 'request_id': 'req-5b3b3a86-dd7e-4912-904c-b319dc407973'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:19:15.338740 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-5b3b3a86-dd7e-4912-904c-b319dc407973 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '238a310adf454901970f67a73b78c4dc', 'request_id': 'req-5b3b3a86-dd7e-4912-904c-b319dc407973'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:19:15.390192 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2d02a4-7256-4899-8089-5cca0175b77d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:19:15.390792 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2d02a4-7256-4899-8089-5cca0175b77d admin admin] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:19:15.391073 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2d02a4-7256-4899-8089-5cca0175b77d admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:19:15.391354 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2d02a4-7256-4899-8089-5cca0175b77d admin admin] PATH_INFO: `/v3/users` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:19:15.392125 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2d02a4-7256-4899-8089-5cca0175b77d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:19:15.394211 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2d02a4-7256-4899-8089-5cca0175b77d admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:19:15.410900 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-5b3b3a86-dd7e-4912-904c-b319dc407973 admin admin] The token cache is being invalidate because project 238a310adf454901970f67a73b78c4dc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80540) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 12:19:15.411258 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-5b3b3a86-dd7e-4912-904c-b319dc407973 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:19:15.411963 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-5b3b3a86-dd7e-4912-904c-b319dc407973 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:19:15.413596 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 969/1914] 10.222.0.23 () {58 vars in 1232 bytes} [Thu May 19 12:19:15 2022] DELETE /identity/v3/projects/238a310adf454901970f67a73b78c4dc => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:19:15.414382 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3b3a86-dd7e-4912-904c-b319dc407973 admin admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:19:15.415750 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-5b3b3a86-dd7e-4912-904c-b319dc407973 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:19:15.422724 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:19:15.428682 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3b3a86-dd7e-4912-904c-b319dc407973 admin admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:19:15.434427 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f75ebb-4b7c-4425-acd5-aff225f37743 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:19:15.435125 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f75ebb-4b7c-4425-acd5-aff225f37743 admin admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:19:15.435417 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f75ebb-4b7c-4425-acd5-aff225f37743 admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:19:15.435703 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f75ebb-4b7c-4425-acd5-aff225f37743 admin admin] PATH_INFO: `/v3/roles` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:19:15.436474 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f75ebb-4b7c-4425-acd5-aff225f37743 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:19:15.438107 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f75ebb-4b7c-4425-acd5-aff225f37743 admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:19:15.449064 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 970/1915] 10.222.0.23 () {58 vars in 1121 bytes} [Thu May 19 12:19:15 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 12:19:15.450332 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8babc1fd-7980-417e-a9fe-917a69a4119f None None] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:19:15.450625 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8babc1fd-7980-417e-a9fe-917a69a4119f None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:19:15.450909 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8babc1fd-7980-417e-a9fe-917a69a4119f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:19:15.468862 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.auth.core [None req-8babc1fd-7980-417e-a9fe-917a69a4119f None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:19:15.470076 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-8babc1fd-7980-417e-a9fe-917a69a4119f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:19:15.478390 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 971/1916] 10.222.0.23 () {58 vars in 957 bytes} [Thu May 19 12:19:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:19:15.479145 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8babc1fd-7980-417e-a9fe-917a69a4119f None None] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:19:15.488584 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8babc1fd-7980-417e-a9fe-917a69a4119f None None] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:19:15.495707 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7652f97a-aded-4702-8da1-1a21826769d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:19:15.495707 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7652f97a-aded-4702-8da1-1a21826769d4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:19:15.495707 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7652f97a-aded-4702-8da1-1a21826769d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:19:15.496253 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7652f97a-aded-4702-8da1-1a21826769d4 admin admin] PATH_INFO: `/v3/projects/f9e4da3b1226427ea9894d73f4dcbb06/users/942d2f1333c546d98048f40aa961d3f8/roles/9db5671c65d3414ea2682afa2ccf1403` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:19:15.513317 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7652f97a-aded-4702-8da1-1a21826769d4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f9e4da3b1226427ea9894d73f4dcbb06, user_id=942d2f1333c546d98048f40aa961d3f8, role_id=9db5671c65d3414ea2682afa2ccf1403)` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:19:15.517408 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7652f97a-aded-4702-8da1-1a21826769d4 admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:19:15.533484 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 946/1917] 10.222.0.23 () {60 vars in 1143 bytes} [Thu May 19 12:19:15 2022] POST /identity/v3/users => generated 292 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 12:19:15.533484 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb30430-02e2-4057-8e14-6688d8d6371c None None] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:19:15.533484 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb30430-02e2-4057-8e14-6688d8d6371c None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:19:15.533484 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb30430-02e2-4057-8e14-6688d8d6371c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:19:15.548873 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:19:15.552052 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.auth.core [None req-7fb30430-02e2-4057-8e14-6688d8d6371c None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:19:15.553258 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-7fb30430-02e2-4057-8e14-6688d8d6371c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:19:15.561356 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 947/1918] 10.222.0.23 () {58 vars in 957 bytes} [Thu May 19 12:19:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:19:15.562347 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb30430-02e2-4057-8e14-6688d8d6371c None None] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:19:15.563625 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-7fb30430-02e2-4057-8e14-6688d8d6371c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:19:15.575674 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb30430-02e2-4057-8e14-6688d8d6371c None None] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:19:15.580478 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39779f1c-48dd-4093-a754-6348ca335500 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:19:15.581154 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39779f1c-48dd-4093-a754-6348ca335500 admin admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:19:15.581436 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39779f1c-48dd-4093-a754-6348ca335500 admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:19:15.581710 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39779f1c-48dd-4093-a754-6348ca335500 admin admin] PATH_INFO: `/v3/roles` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:19:15.582446 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39779f1c-48dd-4093-a754-6348ca335500 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:19:15.584176 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39779f1c-48dd-4093-a754-6348ca335500 admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:19:15.599067 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 948/1919] 10.222.0.23 () {58 vars in 1121 bytes} [Thu May 19 12:19:15 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 12:19:15.599067 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39779f1c-48dd-4093-a754-6348ca335500 admin admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:19:15.599357 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 972/1920] 10.222.0.23 () {60 vars in 1482 bytes} [Thu May 19 12:19:15 2022] PUT /identity/v3/projects/f9e4da3b1226427ea9894d73f4dcbb06/users/942d2f1333c546d98048f40aa961d3f8/roles/9db5671c65d3414ea2682afa2ccf1403 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:19:15.603354 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7652f97a-aded-4702-8da1-1a21826769d4 admin admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:19:15.613255 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39779f1c-48dd-4093-a754-6348ca335500 admin admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:19:15.678490 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7652f97a-aded-4702-8da1-1a21826769d4 admin admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:19:15.682217 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa495e56-79b2-4dc8-8d6d-9fd3c99c3d94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:19:15.682895 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa495e56-79b2-4dc8-8d6d-9fd3c99c3d94 admin admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:19:15.683183 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa495e56-79b2-4dc8-8d6d-9fd3c99c3d94 admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:19:15.683455 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa495e56-79b2-4dc8-8d6d-9fd3c99c3d94 admin admin] PATH_INFO: `/v3/domains` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:19:15.684247 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa495e56-79b2-4dc8-8d6d-9fd3c99c3d94 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:19:15.684431 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ca81de-bb91-4b2f-b2b8-e182dcfaf16a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:19:15.684538 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa495e56-79b2-4dc8-8d6d-9fd3c99c3d94 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:19:15.685329 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ca81de-bb91-4b2f-b2b8-e182dcfaf16a admin admin] REQUEST_METHOD: `PUT` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:19:15.685611 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ca81de-bb91-4b2f-b2b8-e182dcfaf16a admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:19:15.685891 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ca81de-bb91-4b2f-b2b8-e182dcfaf16a admin admin] PATH_INFO: `/v3/projects/454cefaf2cf64baa921c66db47f7ece4/users/e60acaae7b074b24bff9ed8298dd0a81/roles/9db5671c65d3414ea2682afa2ccf1403` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:19:15.686141 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa495e56-79b2-4dc8-8d6d-9fd3c99c3d94 admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:19:15.701294 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 949/1921] 10.222.0.23 () {58 vars in 1152 bytes} [Thu May 19 12:19:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:19:15.702362 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa495e56-79b2-4dc8-8d6d-9fd3c99c3d94 admin admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:19:15.713662 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ca81de-bb91-4b2f-b2b8-e182dcfaf16a admin admin] RBAC: Authorizing `identity:create_grant(project_id=454cefaf2cf64baa921c66db47f7ece4, user_id=e60acaae7b074b24bff9ed8298dd0a81, role_id=9db5671c65d3414ea2682afa2ccf1403)` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:19:15.714010 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa495e56-79b2-4dc8-8d6d-9fd3c99c3d94 admin admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:19:15.716896 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ca81de-bb91-4b2f-b2b8-e182dcfaf16a admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:19:15.720334 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59132980-7050-466a-8b4f-857469b8dd6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:19:15.720798 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59132980-7050-466a-8b4f-857469b8dd6c admin admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:19:15.721079 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59132980-7050-466a-8b4f-857469b8dd6c admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:19:15.721360 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59132980-7050-466a-8b4f-857469b8dd6c admin admin] PATH_INFO: `/v3/roles` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:19:15.722094 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59132980-7050-466a-8b4f-857469b8dd6c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:19:15.723684 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59132980-7050-466a-8b4f-857469b8dd6c admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:19:15.736107 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 950/1922] 10.222.0.23 () {58 vars in 1121 bytes} [Thu May 19 12:19:15 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 12:19:15.739673 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59132980-7050-466a-8b4f-857469b8dd6c admin admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:19:15.751205 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59132980-7050-466a-8b4f-857469b8dd6c admin admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:19:15.755967 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ec852d-3a5c-46ef-8031-6c2221433a5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:19:15.756634 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ec852d-3a5c-46ef-8031-6c2221433a5b admin admin] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:19:15.756923 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ec852d-3a5c-46ef-8031-6c2221433a5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:19:15.757318 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ec852d-3a5c-46ef-8031-6c2221433a5b admin admin] PATH_INFO: `/v3/projects` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:19:15.758016 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ec852d-3a5c-46ef-8031-6c2221433a5b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:19:15.760642 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ec852d-3a5c-46ef-8031-6c2221433a5b admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:19:15.763338 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:19:15.811504 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 973/1923] 10.222.0.23 () {60 vars in 1482 bytes} [Thu May 19 12:19:15 2022] PUT /identity/v3/projects/454cefaf2cf64baa921c66db47f7ece4/users/e60acaae7b074b24bff9ed8298dd0a81/roles/9db5671c65d3414ea2682afa2ccf1403 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:19:15.814618 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ca81de-bb91-4b2f-b2b8-e182dcfaf16a admin admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:19:15.880767 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ca81de-bb91-4b2f-b2b8-e182dcfaf16a admin admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:19:15.885980 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c139dba-af28-4458-8ee4-ae82b7c86e2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:19:15.886885 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c139dba-af28-4458-8ee4-ae82b7c86e2a admin admin] REQUEST_METHOD: `PUT` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:19:15.887192 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c139dba-af28-4458-8ee4-ae82b7c86e2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:19:15.887474 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c139dba-af28-4458-8ee4-ae82b7c86e2a admin admin] PATH_INFO: `/v3/projects/f9e4da3b1226427ea9894d73f4dcbb06/users/942d2f1333c546d98048f40aa961d3f8/roles/9db5671c65d3414ea2682afa2ccf1403` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:19:15.889569 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c139dba-af28-4458-8ee4-ae82b7c86e2a admin admin] RBAC: Authorizing `identity:create_grant(project_id=f9e4da3b1226427ea9894d73f4dcbb06, user_id=942d2f1333c546d98048f40aa961d3f8, role_id=9db5671c65d3414ea2682afa2ccf1403)` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:19:15.892342 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c139dba-af28-4458-8ee4-ae82b7c86e2a admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:19:15.904839 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 974/1924] 10.222.0.23 () {60 vars in 1482 bytes} [Thu May 19 12:19:15 2022] PUT /identity/v3/projects/f9e4da3b1226427ea9894d73f4dcbb06/users/942d2f1333c546d98048f40aa961d3f8/roles/9db5671c65d3414ea2682afa2ccf1403 => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:19:15.907418 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c139dba-af28-4458-8ee4-ae82b7c86e2a admin admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:19:15.922574 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 951/1925] 10.222.0.23 () {60 vars in 1152 bytes} [Thu May 19 12:19:15 2022] POST /identity/v3/projects => generated 365 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 12:19:15.924268 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e02a5c5-c3e9-43a6-acc2-fd71cd8116fc None None] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:19:15.925017 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e02a5c5-c3e9-43a6-acc2-fd71cd8116fc None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:19:15.925530 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e02a5c5-c3e9-43a6-acc2-fd71cd8116fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:19:15.956104 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.auth.core [None req-2e02a5c5-c3e9-43a6-acc2-fd71cd8116fc None None] MFA Rules not processed for user `942d2f1333c546d98048f40aa961d3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:19:15.957361 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-2e02a5c5-c3e9-43a6-acc2-fd71cd8116fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:19:15.989135 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:19:15.992588 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c139dba-af28-4458-8ee4-ae82b7c86e2a admin admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:19:16.032288 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 952/1926] 10.222.0.23 () {58 vars in 957 bytes} [Thu May 19 12:19:15 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:19:16.033434 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e02a5c5-c3e9-43a6-acc2-fd71cd8116fc None None] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:19:16.059602 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2e4661-c13e-4b3b-bc70-8eeac9526412 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:19:16.060337 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2e4661-c13e-4b3b-bc70-8eeac9526412 admin admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:19:16.060623 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2e4661-c13e-4b3b-bc70-8eeac9526412 admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:19:16.060898 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2e4661-c13e-4b3b-bc70-8eeac9526412 admin admin] PATH_INFO: `/v3/roles` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:19:16.061826 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2e4661-c13e-4b3b-bc70-8eeac9526412 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:19:16.063457 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2e4661-c13e-4b3b-bc70-8eeac9526412 admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:19:16.070963 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:INSERT=7,keystone:SELECT=93 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:19:16.082312 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 975/1927] 10.222.0.23 () {58 vars in 1121 bytes} [Thu May 19 12:19:15 2022] GET /identity/v3/roles => generated 1420 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 12:19:16.084056 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2e4661-c13e-4b3b-bc70-8eeac9526412 admin admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:19:16.086585 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-ca2e4661-c13e-4b3b-bc70-8eeac9526412 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:19:16.116839 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e02a5c5-c3e9-43a6-acc2-fd71cd8116fc None None] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:19:16.125349 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c36c97-9b02-4047-bd1a-3bd0e3187030 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:19:16.126635 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c36c97-9b02-4047-bd1a-3bd0e3187030 admin admin] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:19:16.127344 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c36c97-9b02-4047-bd1a-3bd0e3187030 admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:19:16.127666 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c36c97-9b02-4047-bd1a-3bd0e3187030 admin admin] PATH_INFO: `/v3/users` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:19:16.131401 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c36c97-9b02-4047-bd1a-3bd0e3187030 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:19:16.132476 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c36c97-9b02-4047-bd1a-3bd0e3187030 admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:19:16.154099 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2e4661-c13e-4b3b-bc70-8eeac9526412 admin admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:19:16.164331 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5c8ab7-1e64-4e81-b312-a94e7af26d84 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8800254c0e784e42a791fb9abc274d79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd70d25bf69314820b8aaecb65576978a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:19:16.164331 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5c8ab7-1e64-4e81-b312-a94e7af26d84 service cinder] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:19:16.164331 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5c8ab7-1e64-4e81-b312-a94e7af26d84 service cinder] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:19:16.164331 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5c8ab7-1e64-4e81-b312-a94e7af26d84 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:19:16.171265 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5c8ab7-1e64-4e81-b312-a94e7af26d84 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:19:16.172786 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5c8ab7-1e64-4e81-b312-a94e7af26d84 service cinder] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:19:16.184792 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 976/1928] 10.222.0.23 () {60 vars in 1365 bytes} [Thu May 19 12:19:16 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:19:16.185627 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5c8ab7-1e64-4e81-b312-a94e7af26d84 service cinder] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:19:16.196601 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5c8ab7-1e64-4e81-b312-a94e7af26d84 service cinder] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:19:16.202273 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dde7300-ced5-4002-b877-95cd5a5385b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:19:16.203654 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dde7300-ced5-4002-b877-95cd5a5385b0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:19:16.204291 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dde7300-ced5-4002-b877-95cd5a5385b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:19:16.204749 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dde7300-ced5-4002-b877-95cd5a5385b0 admin admin] PATH_INFO: `/v3/projects/454cefaf2cf64baa921c66db47f7ece4/users/e60acaae7b074b24bff9ed8298dd0a81/roles/9db5671c65d3414ea2682afa2ccf1403` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:19:16.207056 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dde7300-ced5-4002-b877-95cd5a5385b0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=454cefaf2cf64baa921c66db47f7ece4, user_id=e60acaae7b074b24bff9ed8298dd0a81, role_id=9db5671c65d3414ea2682afa2ccf1403)` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:19:16.210321 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dde7300-ced5-4002-b877-95cd5a5385b0 admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:19:16.222722 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 977/1929] 10.222.0.23 () {60 vars in 1482 bytes} [Thu May 19 12:19:16 2022] PUT /identity/v3/projects/454cefaf2cf64baa921c66db47f7ece4/users/e60acaae7b074b24bff9ed8298dd0a81/roles/9db5671c65d3414ea2682afa2ccf1403 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:19:16.234323 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71206df7-b7ef-4840-8551-726876f3a5d2 None None] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:19:16.242418 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71206df7-b7ef-4840-8551-726876f3a5d2 None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:19:16.242418 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71206df7-b7ef-4840-8551-726876f3a5d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:19:16.264950 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.auth.core [None req-71206df7-b7ef-4840-8551-726876f3a5d2 None None] MFA Rules not processed for user `e60acaae7b074b24bff9ed8298dd0a81`. Rule list: `[]` (Enabled: `True`). {{(pid=80540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:19:16.267108 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-71206df7-b7ef-4840-8551-726876f3a5d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:19:16.270245 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 953/1930] 10.222.0.23 () {60 vars in 1143 bytes} [Thu May 19 12:19:16 2022] POST /identity/v3/users => generated 283 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 12:19:16.271374 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c36c97-9b02-4047-bd1a-3bd0e3187030 admin admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:19:16.282164 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c36c97-9b02-4047-bd1a-3bd0e3187030 admin admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:19:16.298895 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:19:16.353499 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 978/1931] 10.222.0.23 () {58 vars in 957 bytes} [Thu May 19 12:19:16 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 12:19:16.354217 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4521b1cb-39c2-4878-9791-1f4a8423d064 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:19:16.354972 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4521b1cb-39c2-4878-9791-1f4a8423d064 admin admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:19:16.355265 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4521b1cb-39c2-4878-9791-1f4a8423d064 admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:19:16.355764 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4521b1cb-39c2-4878-9791-1f4a8423d064 admin admin] PATH_INFO: `/v3/roles` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:19:16.356380 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4521b1cb-39c2-4878-9791-1f4a8423d064 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:19:16.360675 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4521b1cb-39c2-4878-9791-1f4a8423d064 admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:19:16.365912 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71206df7-b7ef-4840-8551-726876f3a5d2 None None] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:19:16.367423 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-71206df7-b7ef-4840-8551-726876f3a5d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:19:16.376102 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 954/1932] 10.222.0.23 () {58 vars in 1121 bytes} [Thu May 19 12:19:16 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 12:19:16.385127 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4521b1cb-39c2-4878-9791-1f4a8423d064 admin admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:19:16.395749 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4521b1cb-39c2-4878-9791-1f4a8423d064 admin admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:19:16.404166 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49a556a-5c0e-4bb2-92a8-a83dd1ee5a88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:19:16.404840 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49a556a-5c0e-4bb2-92a8-a83dd1ee5a88 admin admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:19:16.405129 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49a556a-5c0e-4bb2-92a8-a83dd1ee5a88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:19:16.405485 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49a556a-5c0e-4bb2-92a8-a83dd1ee5a88 admin admin] PATH_INFO: `/v3/roles` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:19:16.406149 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49a556a-5c0e-4bb2-92a8-a83dd1ee5a88 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:19:16.407776 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49a556a-5c0e-4bb2-92a8-a83dd1ee5a88 admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:19:16.425069 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 955/1933] 10.222.0.23 () {58 vars in 1121 bytes} [Thu May 19 12:19:16 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 12:19:16.428974 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49a556a-5c0e-4bb2-92a8-a83dd1ee5a88 admin admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:19:16.433190 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71206df7-b7ef-4840-8551-726876f3a5d2 None None] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:19:16.437288 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ec8ff6-7b4f-4415-b8f7-3501e7fdfaa6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8800254c0e784e42a791fb9abc274d79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd70d25bf69314820b8aaecb65576978a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:19:16.437955 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ec8ff6-7b4f-4415-b8f7-3501e7fdfaa6 service cinder] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:19:16.438247 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ec8ff6-7b4f-4415-b8f7-3501e7fdfaa6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:19:16.438530 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ec8ff6-7b4f-4415-b8f7-3501e7fdfaa6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:19:16.440433 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49a556a-5c0e-4bb2-92a8-a83dd1ee5a88 admin admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:19:16.440559 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-f2ec8ff6-7b4f-4415-b8f7-3501e7fdfaa6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:19:16.447349 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058e1c18-9da2-486e-92de-bd57c38f43ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:19:16.447684 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058e1c18-9da2-486e-92de-bd57c38f43ec admin admin] REQUEST_METHOD: `PUT` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:19:16.448042 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058e1c18-9da2-486e-92de-bd57c38f43ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:19:16.448325 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058e1c18-9da2-486e-92de-bd57c38f43ec admin admin] PATH_INFO: `/v3/projects/e2fe4c6f9d58457eaac9eab9643b897a/users/94e9dc9a5e53488ebab6d5cc72a9ee3d/roles/259eb400cdb4439db72d14456ae9cd08` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:19:16.466435 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058e1c18-9da2-486e-92de-bd57c38f43ec admin admin] RBAC: Authorizing `identity:create_grant(project_id=e2fe4c6f9d58457eaac9eab9643b897a, user_id=94e9dc9a5e53488ebab6d5cc72a9ee3d, role_id=259eb400cdb4439db72d14456ae9cd08)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:19:16.469155 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058e1c18-9da2-486e-92de-bd57c38f43ec admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:19:16.519195 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ec8ff6-7b4f-4415-b8f7-3501e7fdfaa6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:19:16.521903 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ec8ff6-7b4f-4415-b8f7-3501e7fdfaa6 service cinder] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:19:16.537910 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 979/1934] 10.222.0.23 () {60 vars in 1365 bytes} [Thu May 19 12:19:16 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 12:19:16.621765 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:19:16.627696 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 956/1935] 10.222.0.23 () {60 vars in 1482 bytes} [Thu May 19 12:19:16 2022] PUT /identity/v3/projects/e2fe4c6f9d58457eaac9eab9643b897a/users/94e9dc9a5e53488ebab6d5cc72a9ee3d/roles/259eb400cdb4439db72d14456ae9cd08 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 12:19:16.639678 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e060e59-e852-4e23-82f8-e18bbf184d34 None None] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:19:16.640078 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e060e59-e852-4e23-82f8-e18bbf184d34 None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:19:16.640427 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e060e59-e852-4e23-82f8-e18bbf184d34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:19:16.671909 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.auth.core [None req-1e060e59-e852-4e23-82f8-e18bbf184d34 None None] MFA Rules not processed for user `94e9dc9a5e53488ebab6d5cc72a9ee3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:19:16.673264 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-1e060e59-e852-4e23-82f8-e18bbf184d34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:19:16.758390 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 980/1936] 10.222.0.23 () {58 vars in 957 bytes} [Thu May 19 12:19:16 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 12:19:16.763053 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058e1c18-9da2-486e-92de-bd57c38f43ec admin admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:19:16.831011 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058e1c18-9da2-486e-92de-bd57c38f43ec admin admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:19:16.836587 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1200e0-f4ca-49d9-b7fc-c9865c7e1cdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:19:16.837455 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1200e0-f4ca-49d9-b7fc-c9865c7e1cdd admin admin] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:19:16.837838 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1200e0-f4ca-49d9-b7fc-c9865c7e1cdd admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:19:16.838162 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1200e0-f4ca-49d9-b7fc-c9865c7e1cdd admin admin] PATH_INFO: `/v3/projects` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:19:16.838891 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1200e0-f4ca-49d9-b7fc-c9865c7e1cdd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:19:16.840066 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:19:16.849528 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1200e0-f4ca-49d9-b7fc-c9865c7e1cdd admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:19:16.982511 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 957/1937] 10.222.0.23 () {60 vars in 1152 bytes} [Thu May 19 12:19:16 2022] POST /identity/v3/projects => generated 367 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 12:19:16.984011 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e060e59-e852-4e23-82f8-e18bbf184d34 None None] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:19:16.996769 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e060e59-e852-4e23-82f8-e18bbf184d34 None None] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:19:17.062361 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a991daca-89de-4c7d-a517-348a05535d6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:19:17.063742 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a991daca-89de-4c7d-a517-348a05535d6f admin admin] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:19:17.063742 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a991daca-89de-4c7d-a517-348a05535d6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:19:17.063742 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a991daca-89de-4c7d-a517-348a05535d6f admin admin] PATH_INFO: `/v3/users` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:19:17.064298 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a991daca-89de-4c7d-a517-348a05535d6f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:19:17.066274 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a991daca-89de-4c7d-a517-348a05535d6f admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:19:17.175664 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 981/1938] 10.222.0.23 () {60 vars in 1143 bytes} [Thu May 19 12:19:16 2022] POST /identity/v3/users => generated 290 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 12:19:17.180606 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1200e0-f4ca-49d9-b7fc-c9865c7e1cdd admin admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:19:17.191121 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1200e0-f4ca-49d9-b7fc-c9865c7e1cdd admin admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:19:17.195967 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1057fccc-7d03-47ed-961d-78602bf32d72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:19:17.196640 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1057fccc-7d03-47ed-961d-78602bf32d72 admin admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:19:17.196925 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1057fccc-7d03-47ed-961d-78602bf32d72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:19:17.197204 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1057fccc-7d03-47ed-961d-78602bf32d72 admin admin] PATH_INFO: `/v3/roles` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:19:17.197946 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1057fccc-7d03-47ed-961d-78602bf32d72 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:19:17.199564 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1057fccc-7d03-47ed-961d-78602bf32d72 admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:19:17.210796 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 958/1939] 10.222.0.23 () {58 vars in 1121 bytes} [Thu May 19 12:19:17 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 12:19:17.215430 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a991daca-89de-4c7d-a517-348a05535d6f admin admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:19:17.226469 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a991daca-89de-4c7d-a517-348a05535d6f admin admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:19:17.231804 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1389f092-1909-4be7-b625-66b8b8f80794 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:19:17.232745 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1389f092-1909-4be7-b625-66b8b8f80794 admin admin] REQUEST_METHOD: `PUT` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:19:17.233032 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1389f092-1909-4be7-b625-66b8b8f80794 admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:19:17.233314 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1389f092-1909-4be7-b625-66b8b8f80794 admin admin] PATH_INFO: `/v3/projects/b60a85fe7a4b4c318a54ee92a0a55a82/users/905bbd85d9ff43c2bc2a4846f9ba8673/roles/9db5671c65d3414ea2682afa2ccf1403` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:19:17.250326 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1389f092-1909-4be7-b625-66b8b8f80794 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b60a85fe7a4b4c318a54ee92a0a55a82, user_id=905bbd85d9ff43c2bc2a4846f9ba8673, role_id=9db5671c65d3414ea2682afa2ccf1403)` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:19:17.253066 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1389f092-1909-4be7-b625-66b8b8f80794 admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:19:17.353294 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 982/1940] 10.222.0.23 () {60 vars in 1482 bytes} [Thu May 19 12:19:17 2022] PUT /identity/v3/projects/b60a85fe7a4b4c318a54ee92a0a55a82/users/905bbd85d9ff43c2bc2a4846f9ba8673/roles/9db5671c65d3414ea2682afa2ccf1403 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:19:17.354447 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1057fccc-7d03-47ed-961d-78602bf32d72 admin admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:19:17.368490 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1057fccc-7d03-47ed-961d-78602bf32d72 admin admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:19:17.470201 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc8304e-0bc9-496e-8b79-ede2b4a22098 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:19:17.471210 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc8304e-0bc9-496e-8b79-ede2b4a22098 admin admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:19:17.473995 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc8304e-0bc9-496e-8b79-ede2b4a22098 admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:19:17.474469 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc8304e-0bc9-496e-8b79-ede2b4a22098 admin admin] PATH_INFO: `/v3/roles` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:19:17.475425 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc8304e-0bc9-496e-8b79-ede2b4a22098 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:19:17.482763 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc8304e-0bc9-496e-8b79-ede2b4a22098 admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:19:17.498728 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 959/1941] 10.222.0.23 () {58 vars in 1121 bytes} [Thu May 19 12:19:17 2022] GET /identity/v3/roles => generated 1420 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 12:19:17.503245 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1389f092-1909-4be7-b625-66b8b8f80794 admin admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:19:17.517593 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1389f092-1909-4be7-b625-66b8b8f80794 admin admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:19:17.524685 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe06d434-467c-4546-9118-eaab127f3790 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:19:17.525643 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe06d434-467c-4546-9118-eaab127f3790 admin admin] REQUEST_METHOD: `PUT` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:19:17.525933 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe06d434-467c-4546-9118-eaab127f3790 admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:19:17.526215 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe06d434-467c-4546-9118-eaab127f3790 admin admin] PATH_INFO: `/v3/projects/b60a85fe7a4b4c318a54ee92a0a55a82/users/905bbd85d9ff43c2bc2a4846f9ba8673/roles/9db5671c65d3414ea2682afa2ccf1403` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:19:17.528541 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe06d434-467c-4546-9118-eaab127f3790 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b60a85fe7a4b4c318a54ee92a0a55a82, user_id=905bbd85d9ff43c2bc2a4846f9ba8673, role_id=9db5671c65d3414ea2682afa2ccf1403)` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:19:17.532091 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe06d434-467c-4546-9118-eaab127f3790 admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:19:17.544034 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 983/1942] 10.222.0.23 () {60 vars in 1482 bytes} [Thu May 19 12:19:17 2022] PUT /identity/v3/projects/b60a85fe7a4b4c318a54ee92a0a55a82/users/905bbd85d9ff43c2bc2a4846f9ba8673/roles/9db5671c65d3414ea2682afa2ccf1403 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:19:17.555018 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aadebee-df53-4924-b7c3-c698d33c39e5 None None] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:19:17.555321 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aadebee-df53-4924-b7c3-c698d33c39e5 None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:19:17.555600 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aadebee-df53-4924-b7c3-c698d33c39e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:19:17.584658 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.auth.core [None req-9aadebee-df53-4924-b7c3-c698d33c39e5 None None] MFA Rules not processed for user `905bbd85d9ff43c2bc2a4846f9ba8673`. Rule list: `[]` (Enabled: `True`). {{(pid=80539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:19:17.586864 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-9aadebee-df53-4924-b7c3-c698d33c39e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:19:17.592464 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:19:17.669004 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:19:17.680184 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 960/1943] 10.222.0.23 () {58 vars in 957 bytes} [Thu May 19 12:19:17 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:19:17.692603 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe06d434-467c-4546-9118-eaab127f3790 admin admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:19:17.748851 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aadebee-df53-4924-b7c3-c698d33c39e5 None None] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:19:17.754263 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe06d434-467c-4546-9118-eaab127f3790 admin admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:19:17.758045 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef31a51-ffcd-42da-ac2b-a0c8ced99155 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8800254c0e784e42a791fb9abc274d79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd70d25bf69314820b8aaecb65576978a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:19:17.758797 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef31a51-ffcd-42da-ac2b-a0c8ced99155 service cinder] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:19:17.759403 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef31a51-ffcd-42da-ac2b-a0c8ced99155 service cinder] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:19:17.759403 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef31a51-ffcd-42da-ac2b-a0c8ced99155 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:19:17.769658 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef31a51-ffcd-42da-ac2b-a0c8ced99155 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:19:17.771132 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef31a51-ffcd-42da-ac2b-a0c8ced99155 service cinder] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:19:17.783069 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 984/1944] 10.222.0.23 () {60 vars in 1365 bytes} [Thu May 19 12:19:17 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:19:17.823260 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aadebee-df53-4924-b7c3-c698d33c39e5 None None] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:19:17.828854 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9c3e3d-d14d-4da9-9d35-16f46798c888 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:19:17.829690 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9c3e3d-d14d-4da9-9d35-16f46798c888 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:19:17.829977 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9c3e3d-d14d-4da9-9d35-16f46798c888 admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:19:17.830256 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9c3e3d-d14d-4da9-9d35-16f46798c888 admin admin] PATH_INFO: `/v3/users/a9166286774546a496bae346f0af332d` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:19:17.831510 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9c3e3d-d14d-4da9-9d35-16f46798c888 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a9166286774546a496bae346f0af332d)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:19:17.833753 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9c3e3d-d14d-4da9-9d35-16f46798c888 admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:19:18.109829 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef31a51-ffcd-42da-ac2b-a0c8ced99155 service cinder] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:19:18.119894 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef31a51-ffcd-42da-ac2b-a0c8ced99155 service cinder] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:19:18.123809 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9d3fca-2988-4d89-bbb5-8cc3023c24bc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8800254c0e784e42a791fb9abc274d79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd70d25bf69314820b8aaecb65576978a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:19:18.124486 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9d3fca-2988-4d89-bbb5-8cc3023c24bc service cinder] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:19:18.124773 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9d3fca-2988-4d89-bbb5-8cc3023c24bc service cinder] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:19:18.125056 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9d3fca-2988-4d89-bbb5-8cc3023c24bc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:19:18.153930 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:19:18.190785 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9d3fca-2988-4d89-bbb5-8cc3023c24bc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:19:18.192520 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9d3fca-2988-4d89-bbb5-8cc3023c24bc service cinder] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:19:18.198084 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-ce9c3e3d-d14d-4da9-9d35-16f46798c888 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a9166286774546a496bae346f0af332d'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:19:18.205578 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 985/1945] 10.222.0.23 () {60 vars in 1365 bytes} [Thu May 19 12:19:18 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 12:19:18.213351 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-ce9c3e3d-d14d-4da9-9d35-16f46798c888 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a9166286774546a496bae346f0af332d'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:19:18.214200 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-ce9c3e3d-d14d-4da9-9d35-16f46798c888 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a9166286774546a496bae346f0af332d'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:19:18.324034 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-ce9c3e3d-d14d-4da9-9d35-16f46798c888 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a9166286774546a496bae346f0af332d'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:19:18.332267 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-ce9c3e3d-d14d-4da9-9d35-16f46798c888 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a9166286774546a496bae346f0af332d'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:19:18.338620 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 961/1946] 10.222.0.23 () {58 vars in 1223 bytes} [Thu May 19 12:19:17 2022] DELETE /identity/v3/users/a9166286774546a496bae346f0af332d => generated 0 bytes in 590 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:19:18.343284 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b02e71-0a15-4598-a5cd-e20cce761a8d None None] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:19:18.343579 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b02e71-0a15-4598-a5cd-e20cce761a8d None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:19:18.343903 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b02e71-0a15-4598-a5cd-e20cce761a8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:19:18.368609 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.auth.core [None req-f8b02e71-0a15-4598-a5cd-e20cce761a8d None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:19:18.370690 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-f8b02e71-0a15-4598-a5cd-e20cce761a8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:19:18.470494 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 986/1947] 10.222.0.23 () {58 vars in 957 bytes} [Thu May 19 12:19:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:19:18.484034 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9c3e3d-d14d-4da9-9d35-16f46798c888 admin admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:19:18.485527 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-ce9c3e3d-d14d-4da9-9d35-16f46798c888 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:19:18.557657 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9c3e3d-d14d-4da9-9d35-16f46798c888 admin admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:19:18.563258 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e7ad43-aece-40cf-92cf-69bd17024a8a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a00b3fa7497456895e5e0bf105c4fc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd70d25bf69314820b8aaecb65576978a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:19:18.564158 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e7ad43-aece-40cf-92cf-69bd17024a8a service neutron] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:19:18.564324 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e7ad43-aece-40cf-92cf-69bd17024a8a service neutron] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:19:18.564599 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e7ad43-aece-40cf-92cf-69bd17024a8a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:19:18.577234 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e7ad43-aece-40cf-92cf-69bd17024a8a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:19:18.579408 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e7ad43-aece-40cf-92cf-69bd17024a8a service neutron] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:19:18.591851 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 962/1948] 10.222.0.23 () {60 vars in 1365 bytes} [Thu May 19 12:19:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:19:19.169054 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b02e71-0a15-4598-a5cd-e20cce761a8d None None] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:19:19.170714 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-f8b02e71-0a15-4598-a5cd-e20cce761a8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:19:19.189880 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b02e71-0a15-4598-a5cd-e20cce761a8d None None] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:19:19.215527 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d0fb49-5d9e-4522-9fbd-78927344e038 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:19:19.220318 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d0fb49-5d9e-4522-9fbd-78927344e038 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:19:19.220610 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d0fb49-5d9e-4522-9fbd-78927344e038 admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:19:19.220897 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d0fb49-5d9e-4522-9fbd-78927344e038 admin admin] PATH_INFO: `/v3/projects/3e55f1346ff74acc9aa28661a7f079d5` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:19:19.224579 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d0fb49-5d9e-4522-9fbd-78927344e038 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3e55f1346ff74acc9aa28661a7f079d5)` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:19:19.237656 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d0fb49-5d9e-4522-9fbd-78927344e038 admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:19:19.490275 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-62d0fb49-5d9e-4522-9fbd-78927344e038 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3e55f1346ff74acc9aa28661a7f079d5', 'request_id': 'req-62d0fb49-5d9e-4522-9fbd-78927344e038'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:19:19.498645 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-62d0fb49-5d9e-4522-9fbd-78927344e038 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3e55f1346ff74acc9aa28661a7f079d5', 'request_id': 'req-62d0fb49-5d9e-4522-9fbd-78927344e038'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:19:19.504822 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-62d0fb49-5d9e-4522-9fbd-78927344e038 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3e55f1346ff74acc9aa28661a7f079d5', 'request_id': 'req-62d0fb49-5d9e-4522-9fbd-78927344e038'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:19:19.622316 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-62d0fb49-5d9e-4522-9fbd-78927344e038 admin admin] The token cache is being invalidate because project 3e55f1346ff74acc9aa28661a7f079d5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80540) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 12:19:19.622602 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-62d0fb49-5d9e-4522-9fbd-78927344e038 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:19:19.623392 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-62d0fb49-5d9e-4522-9fbd-78927344e038 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:19:19.624979 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 987/1949] 10.222.0.23 () {58 vars in 1232 bytes} [Thu May 19 12:19:19 2022] DELETE /identity/v3/projects/3e55f1346ff74acc9aa28661a7f079d5 => generated 0 bytes in 457 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:19:19.629086 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e7ad43-aece-40cf-92cf-69bd17024a8a service neutron] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:19:19.640671 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e7ad43-aece-40cf-92cf-69bd17024a8a service neutron] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:19:19.653790 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:19:19.705291 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c906d47f-6ec4-4312-912f-a3fd03adf8ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:19:19.706213 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c906d47f-6ec4-4312-912f-a3fd03adf8ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:19:19.706499 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c906d47f-6ec4-4312-912f-a3fd03adf8ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:19:19.706777 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c906d47f-6ec4-4312-912f-a3fd03adf8ac admin admin] PATH_INFO: `/v3/users/942d2f1333c546d98048f40aa961d3f8` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:19:19.708168 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c906d47f-6ec4-4312-912f-a3fd03adf8ac admin admin] RBAC: Authorizing `identity:delete_user(user_id=942d2f1333c546d98048f40aa961d3f8)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:19:19.710160 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c906d47f-6ec4-4312-912f-a3fd03adf8ac admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:19:20.008242 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-c906d47f-6ec4-4312-912f-a3fd03adf8ac admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '942d2f1333c546d98048f40aa961d3f8'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:19:20.022089 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-c906d47f-6ec4-4312-912f-a3fd03adf8ac admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '942d2f1333c546d98048f40aa961d3f8'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:19:20.023410 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-c906d47f-6ec4-4312-912f-a3fd03adf8ac admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '942d2f1333c546d98048f40aa961d3f8'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:19:20.153371 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-c906d47f-6ec4-4312-912f-a3fd03adf8ac admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '942d2f1333c546d98048f40aa961d3f8'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:19:20.164402 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-c906d47f-6ec4-4312-912f-a3fd03adf8ac admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '942d2f1333c546d98048f40aa961d3f8'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:19:20.182966 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 963/1950] 10.222.0.23 () {58 vars in 1223 bytes} [Thu May 19 12:19:19 2022] DELETE /identity/v3/users/942d2f1333c546d98048f40aa961d3f8 => generated 0 bytes in 553 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:19:20.824035 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d0fb49-5d9e-4522-9fbd-78927344e038 admin admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:19:20.825728 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-62d0fb49-5d9e-4522-9fbd-78927344e038 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:19:20.839762 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d0fb49-5d9e-4522-9fbd-78927344e038 admin admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:19:20.903703 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:19:20.953333 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68de4e74-9845-42dc-b836-6bdb11c2e2b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:19:20.954576 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68de4e74-9845-42dc-b836-6bdb11c2e2b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:19:20.955002 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68de4e74-9845-42dc-b836-6bdb11c2e2b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:19:20.955444 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68de4e74-9845-42dc-b836-6bdb11c2e2b4 admin admin] PATH_INFO: `/v3/projects/f9e4da3b1226427ea9894d73f4dcbb06` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:19:20.957992 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68de4e74-9845-42dc-b836-6bdb11c2e2b4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f9e4da3b1226427ea9894d73f4dcbb06)` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:19:20.962494 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68de4e74-9845-42dc-b836-6bdb11c2e2b4 admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:19:21.112918 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-68de4e74-9845-42dc-b836-6bdb11c2e2b4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f9e4da3b1226427ea9894d73f4dcbb06', 'request_id': 'req-68de4e74-9845-42dc-b836-6bdb11c2e2b4'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:19:21.119142 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-68de4e74-9845-42dc-b836-6bdb11c2e2b4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f9e4da3b1226427ea9894d73f4dcbb06', 'request_id': 'req-68de4e74-9845-42dc-b836-6bdb11c2e2b4'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:19:21.124633 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-68de4e74-9845-42dc-b836-6bdb11c2e2b4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f9e4da3b1226427ea9894d73f4dcbb06', 'request_id': 'req-68de4e74-9845-42dc-b836-6bdb11c2e2b4'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:19:21.233834 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-68de4e74-9845-42dc-b836-6bdb11c2e2b4 admin admin] The token cache is being invalidate because project f9e4da3b1226427ea9894d73f4dcbb06 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80540) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 12:19:21.240246 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-68de4e74-9845-42dc-b836-6bdb11c2e2b4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:19:21.240246 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-68de4e74-9845-42dc-b836-6bdb11c2e2b4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:19:21.240246 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 988/1951] 10.222.0.23 () {58 vars in 1232 bytes} [Thu May 19 12:19:20 2022] DELETE /identity/v3/projects/f9e4da3b1226427ea9894d73f4dcbb06 => generated 0 bytes in 413 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:19:21.244272 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ec146b-76f2-4a51-9718-632a7b7d4aae None None] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:19:21.244766 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ec146b-76f2-4a51-9718-632a7b7d4aae None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:19:21.244766 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ec146b-76f2-4a51-9718-632a7b7d4aae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:19:21.270571 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.auth.core [None req-68ec146b-76f2-4a51-9718-632a7b7d4aae None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:19:21.271777 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-68ec146b-76f2-4a51-9718-632a7b7d4aae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:19:21.340329 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 964/1952] 10.222.0.23 () {58 vars in 957 bytes} [Thu May 19 12:19:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:19:21.348039 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53c81c3-b1d8-4e00-b5ca-77aeafa757c1 None None] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:19:21.348334 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53c81c3-b1d8-4e00-b5ca-77aeafa757c1 None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:19:21.348634 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53c81c3-b1d8-4e00-b5ca-77aeafa757c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:19:21.368830 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.auth.core [None req-e53c81c3-b1d8-4e00-b5ca-77aeafa757c1 None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:19:21.370116 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-e53c81c3-b1d8-4e00-b5ca-77aeafa757c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:19:21.378143 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 989/1953] 10.222.0.23 () {58 vars in 957 bytes} [Thu May 19 12:19:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:19:21.383075 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ec146b-76f2-4a51-9718-632a7b7d4aae None None] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:19:21.392867 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ec146b-76f2-4a51-9718-632a7b7d4aae None None] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:19:21.397448 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11196ad-dcc1-487f-9f4a-117f8371b605 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:19:21.398132 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11196ad-dcc1-487f-9f4a-117f8371b605 admin admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:19:21.398422 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11196ad-dcc1-487f-9f4a-117f8371b605 admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:19:21.398700 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11196ad-dcc1-487f-9f4a-117f8371b605 admin admin] PATH_INFO: `/v3/domains` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:19:21.399435 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e11196ad-dcc1-487f-9f4a-117f8371b605 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:19:21.399745 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e11196ad-dcc1-487f-9f4a-117f8371b605 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:19:21.401479 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e11196ad-dcc1-487f-9f4a-117f8371b605 admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:19:21.414191 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 965/1954] 10.222.0.23 () {58 vars in 1152 bytes} [Thu May 19 12:19:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:19:21.418979 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53c81c3-b1d8-4e00-b5ca-77aeafa757c1 None None] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:19:21.428937 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53c81c3-b1d8-4e00-b5ca-77aeafa757c1 None None] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:19:21.435270 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7fd4d0b-4f83-47ba-a219-00af6d50c399 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:19:21.435270 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fd4d0b-4f83-47ba-a219-00af6d50c399 admin admin] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:19:21.435483 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fd4d0b-4f83-47ba-a219-00af6d50c399 admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:19:21.435737 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fd4d0b-4f83-47ba-a219-00af6d50c399 admin admin] PATH_INFO: `/v3/projects` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:19:21.436491 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7fd4d0b-4f83-47ba-a219-00af6d50c399 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:19:21.438655 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7fd4d0b-4f83-47ba-a219-00af6d50c399 admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:19:21.603617 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 990/1955] 10.222.0.23 () {60 vars in 1152 bytes} [Thu May 19 12:19:21 2022] POST /identity/v3/projects => generated 393 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 12:19:21.606391 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11196ad-dcc1-487f-9f4a-117f8371b605 admin admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:19:21.617196 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11196ad-dcc1-487f-9f4a-117f8371b605 admin admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:19:21.676707 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0e563f-21c0-42be-981b-2e8583c81dd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:19:21.677436 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0e563f-21c0-42be-981b-2e8583c81dd8 admin admin] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:19:21.677719 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0e563f-21c0-42be-981b-2e8583c81dd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:19:21.678022 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0e563f-21c0-42be-981b-2e8583c81dd8 admin admin] PATH_INFO: `/v3/users` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:19:21.678707 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0e563f-21c0-42be-981b-2e8583c81dd8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:19:21.681525 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0e563f-21c0-42be-981b-2e8583c81dd8 admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:19:21.783258 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 966/1956] 10.222.0.23 () {60 vars in 1143 bytes} [Thu May 19 12:19:21 2022] POST /identity/v3/users => generated 297 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 12:19:21.787219 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7fd4d0b-4f83-47ba-a219-00af6d50c399 admin admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:19:21.798885 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7fd4d0b-4f83-47ba-a219-00af6d50c399 admin admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:19:21.804942 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20409c68-0078-4bbb-b198-58b128d4400b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:19:21.809803 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20409c68-0078-4bbb-b198-58b128d4400b admin admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:19:21.809803 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20409c68-0078-4bbb-b198-58b128d4400b admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:19:21.809803 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20409c68-0078-4bbb-b198-58b128d4400b admin admin] PATH_INFO: `/v3/roles` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:19:21.809803 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20409c68-0078-4bbb-b198-58b128d4400b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:19:21.809803 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20409c68-0078-4bbb-b198-58b128d4400b admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:19:21.824052 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 991/1957] 10.222.0.23 () {58 vars in 1121 bytes} [Thu May 19 12:19:21 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 12:19:21.825929 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0e563f-21c0-42be-981b-2e8583c81dd8 admin admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:19:21.838509 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0e563f-21c0-42be-981b-2e8583c81dd8 admin admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:19:21.846309 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8ecd6d-dd01-4c5f-af54-82011cf05993 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:19:21.847002 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8ecd6d-dd01-4c5f-af54-82011cf05993 admin admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:19:21.847291 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8ecd6d-dd01-4c5f-af54-82011cf05993 admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:19:21.847575 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8ecd6d-dd01-4c5f-af54-82011cf05993 admin admin] PATH_INFO: `/v3/roles` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:19:21.848360 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8ecd6d-dd01-4c5f-af54-82011cf05993 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:19:21.856790 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8ecd6d-dd01-4c5f-af54-82011cf05993 admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:19:21.870234 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 967/1958] 10.222.0.23 () {58 vars in 1121 bytes} [Thu May 19 12:19:21 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 12:19:21.880672 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20409c68-0078-4bbb-b198-58b128d4400b admin admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:19:21.891930 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20409c68-0078-4bbb-b198-58b128d4400b admin admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:19:21.897478 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a33d7a8-54d4-4637-91e8-51426c1a5e3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:19:21.898457 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a33d7a8-54d4-4637-91e8-51426c1a5e3f admin admin] REQUEST_METHOD: `PUT` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:19:21.898738 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a33d7a8-54d4-4637-91e8-51426c1a5e3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:19:21.899014 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a33d7a8-54d4-4637-91e8-51426c1a5e3f admin admin] PATH_INFO: `/v3/projects/552bcbb8eaa74974a2fc2535c7b2d75a/users/84344433142e473fb8bd12395c406a15/roles/259eb400cdb4439db72d14456ae9cd08` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:19:21.917640 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a33d7a8-54d4-4637-91e8-51426c1a5e3f admin admin] RBAC: Authorizing `identity:create_grant(project_id=552bcbb8eaa74974a2fc2535c7b2d75a, user_id=84344433142e473fb8bd12395c406a15, role_id=259eb400cdb4439db72d14456ae9cd08)` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:19:21.920428 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a33d7a8-54d4-4637-91e8-51426c1a5e3f admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:19:22.017127 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 992/1959] 10.222.0.23 () {60 vars in 1482 bytes} [Thu May 19 12:19:21 2022] PUT /identity/v3/projects/552bcbb8eaa74974a2fc2535c7b2d75a/users/84344433142e473fb8bd12395c406a15/roles/259eb400cdb4439db72d14456ae9cd08 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:19:22.028854 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3082a82-a166-44a3-aa36-536e759d98b8 None None] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:19:22.029180 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3082a82-a166-44a3-aa36-536e759d98b8 None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:19:22.029466 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3082a82-a166-44a3-aa36-536e759d98b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:19:22.059632 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.auth.core [None req-e3082a82-a166-44a3-aa36-536e759d98b8 None None] MFA Rules not processed for user `84344433142e473fb8bd12395c406a15`. Rule list: `[]` (Enabled: `True`). {{(pid=80539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:19:22.061062 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-e3082a82-a166-44a3-aa36-536e759d98b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:19:22.136214 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 968/1960] 10.222.0.23 () {58 vars in 957 bytes} [Thu May 19 12:19:22 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:19:22.140740 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a33d7a8-54d4-4637-91e8-51426c1a5e3f admin admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:19:22.206352 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a33d7a8-54d4-4637-91e8-51426c1a5e3f admin admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:19:22.212028 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9594fb-cb83-44f8-8c0c-c7a3e85ad196 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:19:22.212715 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9594fb-cb83-44f8-8c0c-c7a3e85ad196 admin admin] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:19:22.212996 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9594fb-cb83-44f8-8c0c-c7a3e85ad196 admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:19:22.213276 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9594fb-cb83-44f8-8c0c-c7a3e85ad196 admin admin] PATH_INFO: `/v3/projects` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:19:22.213985 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9594fb-cb83-44f8-8c0c-c7a3e85ad196 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:19:22.215995 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9594fb-cb83-44f8-8c0c-c7a3e85ad196 admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:19:22.320712 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 993/1961] 10.222.0.23 () {60 vars in 1152 bytes} [Thu May 19 12:19:22 2022] POST /identity/v3/projects => generated 391 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 12:19:22.324874 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3082a82-a166-44a3-aa36-536e759d98b8 None None] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:19:22.336251 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3082a82-a166-44a3-aa36-536e759d98b8 None None] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:19:22.394573 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:19:22.395494 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c46d80-ffe9-4ad6-aae1-0f9c98f618dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:19:22.396273 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c46d80-ffe9-4ad6-aae1-0f9c98f618dd admin admin] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:19:22.396586 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c46d80-ffe9-4ad6-aae1-0f9c98f618dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:19:22.396869 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c46d80-ffe9-4ad6-aae1-0f9c98f618dd admin admin] PATH_INFO: `/v3/users` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:19:22.397583 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c46d80-ffe9-4ad6-aae1-0f9c98f618dd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:19:22.399565 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c46d80-ffe9-4ad6-aae1-0f9c98f618dd admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:19:22.497222 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 969/1962] 10.222.0.23 () {60 vars in 1143 bytes} [Thu May 19 12:19:22 2022] POST /identity/v3/users => generated 302 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 12:19:22.501067 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9594fb-cb83-44f8-8c0c-c7a3e85ad196 admin admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:19:22.512391 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9594fb-cb83-44f8-8c0c-c7a3e85ad196 admin admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:19:22.517166 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607e350b-ff88-4855-a29c-2a6a55caf9fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:19:22.517839 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607e350b-ff88-4855-a29c-2a6a55caf9fa admin admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:19:22.518126 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607e350b-ff88-4855-a29c-2a6a55caf9fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:19:22.518407 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607e350b-ff88-4855-a29c-2a6a55caf9fa admin admin] PATH_INFO: `/v3/roles` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:19:22.519137 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607e350b-ff88-4855-a29c-2a6a55caf9fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:19:22.520859 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607e350b-ff88-4855-a29c-2a6a55caf9fa admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:19:22.532824 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 994/1963] 10.222.0.23 () {58 vars in 1121 bytes} [Thu May 19 12:19:22 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 12:19:22.537239 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c46d80-ffe9-4ad6-aae1-0f9c98f618dd admin admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:19:22.548692 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c46d80-ffe9-4ad6-aae1-0f9c98f618dd admin admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:19:22.553508 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3e00fe-26c9-4ae5-8284-2375a0a61ae4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:19:22.554476 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3e00fe-26c9-4ae5-8284-2375a0a61ae4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:19:22.554760 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3e00fe-26c9-4ae5-8284-2375a0a61ae4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:19:22.555042 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3e00fe-26c9-4ae5-8284-2375a0a61ae4 admin admin] PATH_INFO: `/v3/projects/ac1fa41b6de74fd7a4f1583a2bb1ae54/users/805c0e51d9434a5aa509f390865569a6/roles/9db5671c65d3414ea2682afa2ccf1403` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:19:22.571131 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3e00fe-26c9-4ae5-8284-2375a0a61ae4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ac1fa41b6de74fd7a4f1583a2bb1ae54, user_id=805c0e51d9434a5aa509f390865569a6, role_id=9db5671c65d3414ea2682afa2ccf1403)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:19:22.574044 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3e00fe-26c9-4ae5-8284-2375a0a61ae4 admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:19:22.698080 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 970/1964] 10.222.0.23 () {60 vars in 1482 bytes} [Thu May 19 12:19:22 2022] PUT /identity/v3/projects/ac1fa41b6de74fd7a4f1583a2bb1ae54/users/805c0e51d9434a5aa509f390865569a6/roles/9db5671c65d3414ea2682afa2ccf1403 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:19:22.702136 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607e350b-ff88-4855-a29c-2a6a55caf9fa admin admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:19:22.714152 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607e350b-ff88-4855-a29c-2a6a55caf9fa admin admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:19:22.782382 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a5cad9-9f98-4700-be92-95faf5396204 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:19:22.783520 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a5cad9-9f98-4700-be92-95faf5396204 admin admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:19:22.783520 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a5cad9-9f98-4700-be92-95faf5396204 admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:19:22.784031 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a5cad9-9f98-4700-be92-95faf5396204 admin admin] PATH_INFO: `/v3/roles` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:19:22.784716 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a5cad9-9f98-4700-be92-95faf5396204 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:19:22.786344 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a5cad9-9f98-4700-be92-95faf5396204 admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:19:22.802743 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 995/1965] 10.222.0.23 () {58 vars in 1121 bytes} [Thu May 19 12:19:22 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 12:19:22.812546 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3e00fe-26c9-4ae5-8284-2375a0a61ae4 admin admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:19:22.824477 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3e00fe-26c9-4ae5-8284-2375a0a61ae4 admin admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:19:22.832342 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb66bdd-9500-425b-9ead-d4fbe41ab257 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:19:22.834324 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb66bdd-9500-425b-9ead-d4fbe41ab257 admin admin] REQUEST_METHOD: `PUT` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:19:22.834587 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb66bdd-9500-425b-9ead-d4fbe41ab257 admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:19:22.835060 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb66bdd-9500-425b-9ead-d4fbe41ab257 admin admin] PATH_INFO: `/v3/projects/ac1fa41b6de74fd7a4f1583a2bb1ae54/users/805c0e51d9434a5aa509f390865569a6/roles/9db5671c65d3414ea2682afa2ccf1403` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:19:22.839380 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb66bdd-9500-425b-9ead-d4fbe41ab257 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ac1fa41b6de74fd7a4f1583a2bb1ae54, user_id=805c0e51d9434a5aa509f390865569a6, role_id=9db5671c65d3414ea2682afa2ccf1403)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:19:22.845049 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb66bdd-9500-425b-9ead-d4fbe41ab257 admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:19:22.857565 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 971/1966] 10.222.0.23 () {60 vars in 1482 bytes} [Thu May 19 12:19:22 2022] PUT /identity/v3/projects/ac1fa41b6de74fd7a4f1583a2bb1ae54/users/805c0e51d9434a5aa509f390865569a6/roles/9db5671c65d3414ea2682afa2ccf1403 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:19:22.870632 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c14de1-8107-4c7e-98d5-a959e8e40113 None None] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:19:22.871154 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c14de1-8107-4c7e-98d5-a959e8e40113 None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:19:22.871154 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c14de1-8107-4c7e-98d5-a959e8e40113 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:19:22.899142 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.auth.core [None req-f1c14de1-8107-4c7e-98d5-a959e8e40113 None None] MFA Rules not processed for user `805c0e51d9434a5aa509f390865569a6`. Rule list: `[]` (Enabled: `True`). {{(pid=80540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:19:22.900385 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-f1c14de1-8107-4c7e-98d5-a959e8e40113 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:19:23.005104 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 996/1967] 10.222.0.23 () {58 vars in 957 bytes} [Thu May 19 12:19:22 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 12:19:23.020404 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb66bdd-9500-425b-9ead-d4fbe41ab257 admin admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:19:23.022300 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-dcb66bdd-9500-425b-9ead-d4fbe41ab257 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:19:23.034105 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:19:23.093599 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb66bdd-9500-425b-9ead-d4fbe41ab257 admin admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:19:23.097269 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0b693e-3b62-4c80-b21e-f86522a07a75 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8800254c0e784e42a791fb9abc274d79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd70d25bf69314820b8aaecb65576978a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:19:23.097925 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0b693e-3b62-4c80-b21e-f86522a07a75 service cinder] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:19:23.098211 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0b693e-3b62-4c80-b21e-f86522a07a75 service cinder] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:19:23.098489 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0b693e-3b62-4c80-b21e-f86522a07a75 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:19:23.108681 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0b693e-3b62-4c80-b21e-f86522a07a75 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:19:23.110122 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0b693e-3b62-4c80-b21e-f86522a07a75 service cinder] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:19:23.121630 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 972/1968] 10.222.0.23 () {60 vars in 1365 bytes} [Thu May 19 12:19:23 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:19:23.213258 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c14de1-8107-4c7e-98d5-a959e8e40113 None None] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:19:23.214919 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-f1c14de1-8107-4c7e-98d5-a959e8e40113 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:19:23.236183 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c14de1-8107-4c7e-98d5-a959e8e40113 None None] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:19:23.236183 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2fb79de-3337-47d5-92db-a05416fbdacf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8800254c0e784e42a791fb9abc274d79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd70d25bf69314820b8aaecb65576978a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:19:23.236183 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fb79de-3337-47d5-92db-a05416fbdacf service cinder] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:19:23.236183 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fb79de-3337-47d5-92db-a05416fbdacf service cinder] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:19:23.236516 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fb79de-3337-47d5-92db-a05416fbdacf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:19:23.306810 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2fb79de-3337-47d5-92db-a05416fbdacf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:19:23.308470 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2fb79de-3337-47d5-92db-a05416fbdacf service cinder] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:19:23.318858 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 997/1969] 10.222.0.23 () {60 vars in 1365 bytes} [Thu May 19 12:19:23 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:19:23.445467 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:19:28.547858 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0b693e-3b62-4c80-b21e-f86522a07a75 service cinder] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:19:28.552587 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-6b0b693e-3b62-4c80-b21e-f86522a07a75 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:19:28.678667 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0b693e-3b62-4c80-b21e-f86522a07a75 service cinder] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:19:28.685405 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca3a0b8-a9a2-4d25-9b1f-8d04a2cf6f32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:19:28.686464 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca3a0b8-a9a2-4d25-9b1f-8d04a2cf6f32 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:19:28.686758 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca3a0b8-a9a2-4d25-9b1f-8d04a2cf6f32 admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:19:28.687045 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca3a0b8-a9a2-4d25-9b1f-8d04a2cf6f32 admin admin] PATH_INFO: `/v3/users/2985089453f340e0a9bf62d8fc012031` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:19:28.688559 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca3a0b8-a9a2-4d25-9b1f-8d04a2cf6f32 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2985089453f340e0a9bf62d8fc012031)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:19:28.691738 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca3a0b8-a9a2-4d25-9b1f-8d04a2cf6f32 admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:19:28.972596 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-7ca3a0b8-a9a2-4d25-9b1f-8d04a2cf6f32 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2985089453f340e0a9bf62d8fc012031'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:19:28.986879 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-7ca3a0b8-a9a2-4d25-9b1f-8d04a2cf6f32 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2985089453f340e0a9bf62d8fc012031'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:19:28.987700 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-7ca3a0b8-a9a2-4d25-9b1f-8d04a2cf6f32 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2985089453f340e0a9bf62d8fc012031'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:19:29.113218 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-7ca3a0b8-a9a2-4d25-9b1f-8d04a2cf6f32 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2985089453f340e0a9bf62d8fc012031'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:19:29.119371 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-7ca3a0b8-a9a2-4d25-9b1f-8d04a2cf6f32 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2985089453f340e0a9bf62d8fc012031'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:19:29.127325 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 973/1970] 10.222.0.23 () {58 vars in 1223 bytes} [Thu May 19 12:19:28 2022] DELETE /identity/v3/users/2985089453f340e0a9bf62d8fc012031 => generated 0 bytes in 582 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:19:29.132143 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31997c07-8d7b-40b3-a4ac-b883c5f1b37d None None] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:19:29.133469 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31997c07-8d7b-40b3-a4ac-b883c5f1b37d None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:19:29.133556 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31997c07-8d7b-40b3-a4ac-b883c5f1b37d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:19:29.153809 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.auth.core [None req-31997c07-8d7b-40b3-a4ac-b883c5f1b37d None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:19:29.155182 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-31997c07-8d7b-40b3-a4ac-b883c5f1b37d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:19:29.159769 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:19:29.253112 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 998/1971] 10.222.0.23 () {58 vars in 957 bytes} [Thu May 19 12:19:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:19:29.263366 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca3a0b8-a9a2-4d25-9b1f-8d04a2cf6f32 admin admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:19:29.265928 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-7ca3a0b8-a9a2-4d25-9b1f-8d04a2cf6f32 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:19:29.273943 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:19:29.348986 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca3a0b8-a9a2-4d25-9b1f-8d04a2cf6f32 admin admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:19:29.353707 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81841f59-4f3c-477e-96ef-e3d631fb99d8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a00b3fa7497456895e5e0bf105c4fc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd70d25bf69314820b8aaecb65576978a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:19:29.354365 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81841f59-4f3c-477e-96ef-e3d631fb99d8 service neutron] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:19:29.354656 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81841f59-4f3c-477e-96ef-e3d631fb99d8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:19:29.354938 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81841f59-4f3c-477e-96ef-e3d631fb99d8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:19:29.364586 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81841f59-4f3c-477e-96ef-e3d631fb99d8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:19:29.366026 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81841f59-4f3c-477e-96ef-e3d631fb99d8 service neutron] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:19:29.376998 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 974/1972] 10.222.0.23 () {60 vars in 1365 bytes} [Thu May 19 12:19:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:19:29.802469 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31997c07-8d7b-40b3-a4ac-b883c5f1b37d None None] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:19:29.803942 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-31997c07-8d7b-40b3-a4ac-b883c5f1b37d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:19:29.816156 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31997c07-8d7b-40b3-a4ac-b883c5f1b37d None None] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:19:29.821940 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fc274b-418f-4983-ace7-4080163418ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:19:29.822833 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fc274b-418f-4983-ace7-4080163418ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:19:29.823118 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fc274b-418f-4983-ace7-4080163418ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:19:29.823393 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fc274b-418f-4983-ace7-4080163418ce admin admin] PATH_INFO: `/v3/users/94e9dc9a5e53488ebab6d5cc72a9ee3d` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:19:29.824660 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1fc274b-418f-4983-ace7-4080163418ce admin admin] RBAC: Authorizing `identity:delete_user(user_id=94e9dc9a5e53488ebab6d5cc72a9ee3d)` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:19:29.826631 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1fc274b-418f-4983-ace7-4080163418ce admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:19:29.902836 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81841f59-4f3c-477e-96ef-e3d631fb99d8 service neutron] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:19:29.907847 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-81841f59-4f3c-477e-96ef-e3d631fb99d8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:19:29.941379 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81841f59-4f3c-477e-96ef-e3d631fb99d8 service neutron] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:19:29.948992 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b5df62-fbbb-4c08-a475-82bbbb5c117c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:19:29.950328 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b5df62-fbbb-4c08-a475-82bbbb5c117c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:19:29.950774 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b5df62-fbbb-4c08-a475-82bbbb5c117c admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:19:29.951209 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b5df62-fbbb-4c08-a475-82bbbb5c117c admin admin] PATH_INFO: `/v3/projects/9bf39a0b49fc46ab8691bb59fa4901ae` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:19:29.953509 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b5df62-fbbb-4c08-a475-82bbbb5c117c admin admin] RBAC: Authorizing `identity:delete_project(project_id=9bf39a0b49fc46ab8691bb59fa4901ae)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:19:29.955748 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b5df62-fbbb-4c08-a475-82bbbb5c117c admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:19:30.051210 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-b1fc274b-418f-4983-ace7-4080163418ce admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '94e9dc9a5e53488ebab6d5cc72a9ee3d'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:19:30.068135 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-b1fc274b-418f-4983-ace7-4080163418ce admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '94e9dc9a5e53488ebab6d5cc72a9ee3d'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:19:30.068908 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-b1fc274b-418f-4983-ace7-4080163418ce admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '94e9dc9a5e53488ebab6d5cc72a9ee3d'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:19:30.117676 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-d6b5df62-fbbb-4c08-a475-82bbbb5c117c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9bf39a0b49fc46ab8691bb59fa4901ae', 'request_id': 'req-d6b5df62-fbbb-4c08-a475-82bbbb5c117c'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:19:30.124659 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-d6b5df62-fbbb-4c08-a475-82bbbb5c117c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9bf39a0b49fc46ab8691bb59fa4901ae', 'request_id': 'req-d6b5df62-fbbb-4c08-a475-82bbbb5c117c'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:19:30.215639 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-b1fc274b-418f-4983-ace7-4080163418ce admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '94e9dc9a5e53488ebab6d5cc72a9ee3d'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:19:30.234619 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-b1fc274b-418f-4983-ace7-4080163418ce admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '94e9dc9a5e53488ebab6d5cc72a9ee3d'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:19:30.248785 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 999/1973] 10.222.0.23 () {58 vars in 1223 bytes} [Thu May 19 12:19:29 2022] DELETE /identity/v3/users/94e9dc9a5e53488ebab6d5cc72a9ee3d => generated 0 bytes in 447 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:19:30.259047 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a1cb20-9415-4260-be97-21a4b78deccb None None] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:19:30.262003 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a1cb20-9415-4260-be97-21a4b78deccb None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:19:30.262003 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a1cb20-9415-4260-be97-21a4b78deccb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:19:30.287235 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.auth.core [None req-a2a1cb20-9415-4260-be97-21a4b78deccb None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:19:30.288683 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-a2a1cb20-9415-4260-be97-21a4b78deccb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:19:30.368416 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-d6b5df62-fbbb-4c08-a475-82bbbb5c117c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9bf39a0b49fc46ab8691bb59fa4901ae', 'request_id': 'req-d6b5df62-fbbb-4c08-a475-82bbbb5c117c'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:19:30.392865 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-d6b5df62-fbbb-4c08-a475-82bbbb5c117c admin admin] The token cache is being invalidate because project 9bf39a0b49fc46ab8691bb59fa4901ae was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80539) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 12:19:30.393349 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-d6b5df62-fbbb-4c08-a475-82bbbb5c117c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:19:30.394417 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-d6b5df62-fbbb-4c08-a475-82bbbb5c117c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:19:30.397476 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 975/1974] 10.222.0.23 () {58 vars in 1232 bytes} [Thu May 19 12:19:29 2022] DELETE /identity/v3/projects/9bf39a0b49fc46ab8691bb59fa4901ae => generated 0 bytes in 495 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:19:30.402582 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b5df62-fbbb-4c08-a475-82bbbb5c117c admin admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:19:30.409237 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-d6b5df62-fbbb-4c08-a475-82bbbb5c117c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:19:30.436800 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1000/1975] 10.222.0.23 () {58 vars in 957 bytes} [Thu May 19 12:19:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:19:30.438531 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a1cb20-9415-4260-be97-21a4b78deccb None None] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:19:30.442003 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-a2a1cb20-9415-4260-be97-21a4b78deccb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:19:30.496530 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:19:30.503971 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b5df62-fbbb-4c08-a475-82bbbb5c117c admin admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:19:30.515086 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be19003-3614-4a22-a2cd-f44490b58211 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:19:30.517551 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be19003-3614-4a22-a2cd-f44490b58211 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:19:30.517936 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be19003-3614-4a22-a2cd-f44490b58211 admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:19:30.518411 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be19003-3614-4a22-a2cd-f44490b58211 admin admin] PATH_INFO: `/v3/users/e60acaae7b074b24bff9ed8298dd0a81` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:19:30.520558 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be19003-3614-4a22-a2cd-f44490b58211 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e60acaae7b074b24bff9ed8298dd0a81)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:19:30.524853 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be19003-3614-4a22-a2cd-f44490b58211 admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:19:30.563273 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a1cb20-9415-4260-be97-21a4b78deccb None None] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:19:30.570161 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6a7d5c-4f3b-4067-8a1f-8c44f30be7e4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a00b3fa7497456895e5e0bf105c4fc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd70d25bf69314820b8aaecb65576978a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:19:30.571334 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6a7d5c-4f3b-4067-8a1f-8c44f30be7e4 service neutron] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:19:30.571913 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6a7d5c-4f3b-4067-8a1f-8c44f30be7e4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:19:30.572453 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6a7d5c-4f3b-4067-8a1f-8c44f30be7e4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:19:30.588319 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6a7d5c-4f3b-4067-8a1f-8c44f30be7e4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:19:30.589958 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6a7d5c-4f3b-4067-8a1f-8c44f30be7e4 service neutron] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:19:30.602543 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1001/1976] 10.222.0.23 () {60 vars in 1365 bytes} [Thu May 19 12:19:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:19:30.749881 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:19:30.764788 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-9be19003-3614-4a22-a2cd-f44490b58211 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e60acaae7b074b24bff9ed8298dd0a81'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:19:30.780044 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-9be19003-3614-4a22-a2cd-f44490b58211 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e60acaae7b074b24bff9ed8298dd0a81'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:19:30.781810 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-9be19003-3614-4a22-a2cd-f44490b58211 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e60acaae7b074b24bff9ed8298dd0a81'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:19:30.883047 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-9be19003-3614-4a22-a2cd-f44490b58211 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e60acaae7b074b24bff9ed8298dd0a81'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:19:30.895393 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-9be19003-3614-4a22-a2cd-f44490b58211 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e60acaae7b074b24bff9ed8298dd0a81'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:19:30.904899 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 976/1977] 10.222.0.23 () {58 vars in 1223 bytes} [Thu May 19 12:19:30 2022] DELETE /identity/v3/users/e60acaae7b074b24bff9ed8298dd0a81 => generated 0 bytes in 503 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:19:31.131164 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6a7d5c-4f3b-4067-8a1f-8c44f30be7e4 service neutron] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:19:31.136216 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-0a6a7d5c-4f3b-4067-8a1f-8c44f30be7e4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:19:31.153494 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6a7d5c-4f3b-4067-8a1f-8c44f30be7e4 service neutron] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:19:31.259840 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0cac1be-9780-490b-b700-b640ae019ec9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:19:31.265859 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cac1be-9780-490b-b700-b640ae019ec9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:19:31.265859 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cac1be-9780-490b-b700-b640ae019ec9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:19:31.265859 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cac1be-9780-490b-b700-b640ae019ec9 admin admin] PATH_INFO: `/v3/projects/e2fe4c6f9d58457eaac9eab9643b897a` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:19:31.265859 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0cac1be-9780-490b-b700-b640ae019ec9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e2fe4c6f9d58457eaac9eab9643b897a)` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:19:31.265859 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0cac1be-9780-490b-b700-b640ae019ec9 admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:19:31.419920 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-e0cac1be-9780-490b-b700-b640ae019ec9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e2fe4c6f9d58457eaac9eab9643b897a', 'request_id': 'req-e0cac1be-9780-490b-b700-b640ae019ec9'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:19:31.432423 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-e0cac1be-9780-490b-b700-b640ae019ec9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e2fe4c6f9d58457eaac9eab9643b897a', 'request_id': 'req-e0cac1be-9780-490b-b700-b640ae019ec9'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:19:31.440833 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-e0cac1be-9780-490b-b700-b640ae019ec9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e2fe4c6f9d58457eaac9eab9643b897a', 'request_id': 'req-e0cac1be-9780-490b-b700-b640ae019ec9'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:19:31.543277 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-e0cac1be-9780-490b-b700-b640ae019ec9 admin admin] The token cache is being invalidate because project e2fe4c6f9d58457eaac9eab9643b897a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80540) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 12:19:31.543568 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-e0cac1be-9780-490b-b700-b640ae019ec9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:19:31.544287 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-e0cac1be-9780-490b-b700-b640ae019ec9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:19:31.546020 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1002/1978] 10.222.0.23 () {58 vars in 1232 bytes} [Thu May 19 12:19:31 2022] DELETE /identity/v3/projects/e2fe4c6f9d58457eaac9eab9643b897a => generated 0 bytes in 416 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 12:19:31.546366 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be19003-3614-4a22-a2cd-f44490b58211 admin admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:19:31.549956 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0cac1be-9780-490b-b700-b640ae019ec9 admin admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:19:31.554071 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-e0cac1be-9780-490b-b700-b640ae019ec9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:19:31.555192 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-9be19003-3614-4a22-a2cd-f44490b58211 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:19:31.578981 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be19003-3614-4a22-a2cd-f44490b58211 admin admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:19:31.671486 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:19:31.677566 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a18911-a07d-4e73-a481-8cb10c4a8bde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:19:31.678331 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0cac1be-9780-490b-b700-b640ae019ec9 admin admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:19:31.678493 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a18911-a07d-4e73-a481-8cb10c4a8bde admin admin] REQUEST_METHOD: `DELETE` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:19:31.678779 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a18911-a07d-4e73-a481-8cb10c4a8bde admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:19:31.679060 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a18911-a07d-4e73-a481-8cb10c4a8bde admin admin] PATH_INFO: `/v3/projects/454cefaf2cf64baa921c66db47f7ece4` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:19:31.681362 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a18911-a07d-4e73-a481-8cb10c4a8bde admin admin] RBAC: Authorizing `identity:delete_project(project_id=454cefaf2cf64baa921c66db47f7ece4)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:19:31.683833 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a18911-a07d-4e73-a481-8cb10c4a8bde admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:19:31.686724 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4784878-42a6-4511-bbdf-5b1e3c309833 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:19:31.687895 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4784878-42a6-4511-bbdf-5b1e3c309833 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:19:31.688334 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4784878-42a6-4511-bbdf-5b1e3c309833 admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:19:31.688706 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4784878-42a6-4511-bbdf-5b1e3c309833 admin admin] PATH_INFO: `/v3/users/905bbd85d9ff43c2bc2a4846f9ba8673` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:19:31.690102 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4784878-42a6-4511-bbdf-5b1e3c309833 admin admin] RBAC: Authorizing `identity:delete_user(user_id=905bbd85d9ff43c2bc2a4846f9ba8673)` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:19:31.692687 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4784878-42a6-4511-bbdf-5b1e3c309833 admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:19:31.997694 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-a4784878-42a6-4511-bbdf-5b1e3c309833 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '905bbd85d9ff43c2bc2a4846f9ba8673'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:19:31.997996 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-94a18911-a07d-4e73-a481-8cb10c4a8bde admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '454cefaf2cf64baa921c66db47f7ece4', 'request_id': 'req-94a18911-a07d-4e73-a481-8cb10c4a8bde'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:19:32.004896 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-94a18911-a07d-4e73-a481-8cb10c4a8bde admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '454cefaf2cf64baa921c66db47f7ece4', 'request_id': 'req-94a18911-a07d-4e73-a481-8cb10c4a8bde'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:19:32.010643 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-a4784878-42a6-4511-bbdf-5b1e3c309833 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '905bbd85d9ff43c2bc2a4846f9ba8673'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:19:32.011372 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-a4784878-42a6-4511-bbdf-5b1e3c309833 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '905bbd85d9ff43c2bc2a4846f9ba8673'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:19:32.079206 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-94a18911-a07d-4e73-a481-8cb10c4a8bde admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '454cefaf2cf64baa921c66db47f7ece4', 'request_id': 'req-94a18911-a07d-4e73-a481-8cb10c4a8bde'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:19:32.090667 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-94a18911-a07d-4e73-a481-8cb10c4a8bde admin admin] The token cache is being invalidate because project 454cefaf2cf64baa921c66db47f7ece4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80539) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 12:19:32.090948 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-94a18911-a07d-4e73-a481-8cb10c4a8bde admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:19:32.091578 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-94a18911-a07d-4e73-a481-8cb10c4a8bde admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:19:32.093161 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 977/1979] 10.222.0.23 () {58 vars in 1232 bytes} [Thu May 19 12:19:31 2022] DELETE /identity/v3/projects/454cefaf2cf64baa921c66db47f7ece4 => generated 0 bytes in 548 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 12:19:32.097777 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49839c45-7fb6-4914-b8d2-e1ae33f9b4b3 None None] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:19:32.098064 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49839c45-7fb6-4914-b8d2-e1ae33f9b4b3 None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:19:32.098345 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49839c45-7fb6-4914-b8d2-e1ae33f9b4b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:19:32.123805 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.auth.core [None req-49839c45-7fb6-4914-b8d2-e1ae33f9b4b3 None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:19:32.123805 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-49839c45-7fb6-4914-b8d2-e1ae33f9b4b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:19:32.147592 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-a4784878-42a6-4511-bbdf-5b1e3c309833 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '905bbd85d9ff43c2bc2a4846f9ba8673'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:19:32.152831 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:19:32.155619 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-a4784878-42a6-4511-bbdf-5b1e3c309833 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '905bbd85d9ff43c2bc2a4846f9ba8673'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:19:32.162214 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1003/1980] 10.222.0.23 () {58 vars in 1223 bytes} [Thu May 19 12:19:31 2022] DELETE /identity/v3/users/905bbd85d9ff43c2bc2a4846f9ba8673 => generated 0 bytes in 612 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:19:32.188375 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 978/1981] 10.222.0.23 () {58 vars in 957 bytes} [Thu May 19 12:19:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 12:19:32.192898 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a4526a-5ba1-429a-8716-ec235beddb35 None None] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:19:32.192898 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a4526a-5ba1-429a-8716-ec235beddb35 None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:19:32.192898 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a4526a-5ba1-429a-8716-ec235beddb35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:19:32.212183 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.auth.core [None req-41a4526a-5ba1-429a-8716-ec235beddb35 None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:19:32.227603 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-41a4526a-5ba1-429a-8716-ec235beddb35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:19:32.286287 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1004/1982] 10.222.0.23 () {58 vars in 957 bytes} [Thu May 19 12:19:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:19:32.291623 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49839c45-7fb6-4914-b8d2-e1ae33f9b4b3 None None] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:19:32.301612 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49839c45-7fb6-4914-b8d2-e1ae33f9b4b3 None None] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:19:32.307450 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f08d2ad-139a-4073-8be0-2071521b54a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:19:32.308158 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f08d2ad-139a-4073-8be0-2071521b54a3 admin admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:19:32.308447 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f08d2ad-139a-4073-8be0-2071521b54a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:19:32.308735 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f08d2ad-139a-4073-8be0-2071521b54a3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:19:32.309477 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f08d2ad-139a-4073-8be0-2071521b54a3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:19:32.309790 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f08d2ad-139a-4073-8be0-2071521b54a3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:19:32.311401 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f08d2ad-139a-4073-8be0-2071521b54a3 admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:19:32.324396 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 979/1983] 10.222.0.23 () {58 vars in 1152 bytes} [Thu May 19 12:19:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:19:32.328920 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a4526a-5ba1-429a-8716-ec235beddb35 None None] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:19:32.339012 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a4526a-5ba1-429a-8716-ec235beddb35 None None] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:19:32.344224 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747c4341-6f95-45cf-b803-f0de2cceac13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:19:32.345005 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747c4341-6f95-45cf-b803-f0de2cceac13 admin admin] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:19:32.345287 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747c4341-6f95-45cf-b803-f0de2cceac13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:19:32.345562 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747c4341-6f95-45cf-b803-f0de2cceac13 admin admin] PATH_INFO: `/v3/projects` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:19:32.346264 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747c4341-6f95-45cf-b803-f0de2cceac13 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:19:32.348254 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747c4341-6f95-45cf-b803-f0de2cceac13 admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:19:32.458213 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1005/1984] 10.222.0.23 () {60 vars in 1152 bytes} [Thu May 19 12:19:32 2022] POST /identity/v3/projects => generated 373 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 12:19:32.466025 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f08d2ad-139a-4073-8be0-2071521b54a3 admin admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:19:32.477896 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f08d2ad-139a-4073-8be0-2071521b54a3 admin admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:19:32.542593 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d95c447-1d13-4733-8238-4ffc43f79911 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:19:32.543391 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d95c447-1d13-4733-8238-4ffc43f79911 admin admin] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:19:32.543696 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d95c447-1d13-4733-8238-4ffc43f79911 admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:19:32.544006 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d95c447-1d13-4733-8238-4ffc43f79911 admin admin] PATH_INFO: `/v3/users` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:19:32.544711 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d95c447-1d13-4733-8238-4ffc43f79911 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:19:32.546772 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d95c447-1d13-4733-8238-4ffc43f79911 admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:19:32.684174 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 980/1985] 10.222.0.23 () {60 vars in 1143 bytes} [Thu May 19 12:19:32 2022] POST /identity/v3/users => generated 287 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 12:19:32.684374 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747c4341-6f95-45cf-b803-f0de2cceac13 admin admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:19:32.684567 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-747c4341-6f95-45cf-b803-f0de2cceac13 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:19:32.685201 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d95c447-1d13-4733-8238-4ffc43f79911 admin admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:19:32.700150 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d95c447-1d13-4733-8238-4ffc43f79911 admin admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:19:32.700547 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747c4341-6f95-45cf-b803-f0de2cceac13 admin admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:19:32.707395 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c828b239-636e-4574-a4db-5d09d0d96f29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:19:32.708350 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57872220-b4f3-4786-b01d-cee8d2b629ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:19:32.710249 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c828b239-636e-4574-a4db-5d09d0d96f29 admin admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:19:32.711071 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c828b239-636e-4574-a4db-5d09d0d96f29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:19:32.715896 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57872220-b4f3-4786-b01d-cee8d2b629ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:19:32.715896 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57872220-b4f3-4786-b01d-cee8d2b629ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:19:32.715896 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57872220-b4f3-4786-b01d-cee8d2b629ca admin admin] PATH_INFO: `/v3/projects/b60a85fe7a4b4c318a54ee92a0a55a82` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:19:32.716708 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c828b239-636e-4574-a4db-5d09d0d96f29 admin admin] PATH_INFO: `/v3/roles` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:19:32.716708 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c828b239-636e-4574-a4db-5d09d0d96f29 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:19:32.717438 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57872220-b4f3-4786-b01d-cee8d2b629ca admin admin] RBAC: Authorizing `identity:delete_project(project_id=b60a85fe7a4b4c318a54ee92a0a55a82)` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:19:32.719964 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57872220-b4f3-4786-b01d-cee8d2b629ca admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:19:32.721021 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c828b239-636e-4574-a4db-5d09d0d96f29 admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:19:32.732558 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 981/1986] 10.222.0.23 () {58 vars in 1121 bytes} [Thu May 19 12:19:32 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 19 12:19:32.736650 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c828b239-636e-4574-a4db-5d09d0d96f29 admin admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:19:32.747829 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c828b239-636e-4574-a4db-5d09d0d96f29 admin admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:19:32.754292 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f07bfb-ec54-4374-865a-5dab24475fa0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:19:32.755184 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f07bfb-ec54-4374-865a-5dab24475fa0 admin admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:19:32.755475 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f07bfb-ec54-4374-865a-5dab24475fa0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:19:32.755862 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f07bfb-ec54-4374-865a-5dab24475fa0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:19:32.756584 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f07bfb-ec54-4374-865a-5dab24475fa0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:19:32.767266 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f07bfb-ec54-4374-865a-5dab24475fa0 admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:19:32.787156 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 982/1987] 10.222.0.23 () {58 vars in 1121 bytes} [Thu May 19 12:19:32 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 12:19:32.792053 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f07bfb-ec54-4374-865a-5dab24475fa0 admin admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:19:32.802560 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f07bfb-ec54-4374-865a-5dab24475fa0 admin admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:19:32.808406 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9014708b-1305-4beb-8c2a-874d7d77ef3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:19:32.809311 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9014708b-1305-4beb-8c2a-874d7d77ef3b admin admin] REQUEST_METHOD: `PUT` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:19:32.809593 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9014708b-1305-4beb-8c2a-874d7d77ef3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:19:32.809870 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9014708b-1305-4beb-8c2a-874d7d77ef3b admin admin] PATH_INFO: `/v3/projects/79e70c946e5440e39ee1d7cbeb0a4160/users/858ef1572f064c248592e92625b228f8/roles/259eb400cdb4439db72d14456ae9cd08` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:19:32.825884 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9014708b-1305-4beb-8c2a-874d7d77ef3b admin admin] RBAC: Authorizing `identity:create_grant(project_id=79e70c946e5440e39ee1d7cbeb0a4160, user_id=858ef1572f064c248592e92625b228f8, role_id=259eb400cdb4439db72d14456ae9cd08)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:19:32.828624 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9014708b-1305-4beb-8c2a-874d7d77ef3b admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:19:32.849261 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:19:32.850547 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-57872220-b4f3-4786-b01d-cee8d2b629ca admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b60a85fe7a4b4c318a54ee92a0a55a82', 'request_id': 'req-57872220-b4f3-4786-b01d-cee8d2b629ca'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:19:32.858728 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-57872220-b4f3-4786-b01d-cee8d2b629ca admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b60a85fe7a4b4c318a54ee92a0a55a82', 'request_id': 'req-57872220-b4f3-4786-b01d-cee8d2b629ca'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:19:32.868883 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-57872220-b4f3-4786-b01d-cee8d2b629ca admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b60a85fe7a4b4c318a54ee92a0a55a82', 'request_id': 'req-57872220-b4f3-4786-b01d-cee8d2b629ca'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:19:32.931396 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 983/1988] 10.222.0.23 () {60 vars in 1482 bytes} [Thu May 19 12:19:32 2022] PUT /identity/v3/projects/79e70c946e5440e39ee1d7cbeb0a4160/users/858ef1572f064c248592e92625b228f8/roles/259eb400cdb4439db72d14456ae9cd08 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 12:19:32.938064 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae63aa64-658c-443a-88a0-c496a716ea81 None None] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:19:32.938349 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae63aa64-658c-443a-88a0-c496a716ea81 None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:19:32.938678 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae63aa64-658c-443a-88a0-c496a716ea81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:19:32.958839 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.auth.core [None req-ae63aa64-658c-443a-88a0-c496a716ea81 None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:19:32.960282 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-ae63aa64-658c-443a-88a0-c496a716ea81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:19:32.968504 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-57872220-b4f3-4786-b01d-cee8d2b629ca admin admin] The token cache is being invalidate because project b60a85fe7a4b4c318a54ee92a0a55a82 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80540) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 12:19:32.968846 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-57872220-b4f3-4786-b01d-cee8d2b629ca admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:19:32.969521 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-57872220-b4f3-4786-b01d-cee8d2b629ca admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:19:32.971354 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1006/1989] 10.222.0.23 () {58 vars in 1232 bytes} [Thu May 19 12:19:32 2022] DELETE /identity/v3/projects/b60a85fe7a4b4c318a54ee92a0a55a82 => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:19:32.976180 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817b3d19-2ff8-4859-a092-483b283e2561 None None] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:19:32.976579 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817b3d19-2ff8-4859-a092-483b283e2561 None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:19:32.976938 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817b3d19-2ff8-4859-a092-483b283e2561 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:19:32.996831 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.auth.core [None req-817b3d19-2ff8-4859-a092-483b283e2561 None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:19:32.998226 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-817b3d19-2ff8-4859-a092-483b283e2561 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:19:33.032566 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 984/1990] 10.222.0.23 () {58 vars in 957 bytes} [Thu May 19 12:19:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 12:19:33.043514 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae63aa64-658c-443a-88a0-c496a716ea81 None None] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:19:33.045107 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-ae63aa64-658c-443a-88a0-c496a716ea81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:19:33.058710 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:19:33.066070 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1007/1991] 10.222.0.23 () {58 vars in 957 bytes} [Thu May 19 12:19:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 12:19:33.074418 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2f5b0d-cea7-449c-a7c7-b2052183fd59 None None] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:19:33.074715 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2f5b0d-cea7-449c-a7c7-b2052183fd59 None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:19:33.074998 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2f5b0d-cea7-449c-a7c7-b2052183fd59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:19:33.094316 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.auth.core [None req-4d2f5b0d-cea7-449c-a7c7-b2052183fd59 None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:19:33.099157 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-4d2f5b0d-cea7-449c-a7c7-b2052183fd59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:19:33.105838 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1008/1992] 10.222.0.23 () {58 vars in 957 bytes} [Thu May 19 12:19:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 12:19:33.106001 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae63aa64-658c-443a-88a0-c496a716ea81 None None] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:19:33.110067 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e143993-e5f4-465e-9543-a8a41dfd5bde service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a00b3fa7497456895e5e0bf105c4fc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd70d25bf69314820b8aaecb65576978a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:19:33.110229 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2f5b0d-cea7-449c-a7c7-b2052183fd59 None None] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:19:33.110832 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e143993-e5f4-465e-9543-a8a41dfd5bde service neutron] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:19:33.111110 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e143993-e5f4-465e-9543-a8a41dfd5bde service neutron] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:19:33.111391 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e143993-e5f4-465e-9543-a8a41dfd5bde service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:19:33.119574 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2f5b0d-cea7-449c-a7c7-b2052183fd59 None None] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:19:33.123277 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e143993-e5f4-465e-9543-a8a41dfd5bde service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:19:33.129677 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44093815-890b-4cc4-b543-4ff25ee4fa5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:19:33.129832 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e143993-e5f4-465e-9543-a8a41dfd5bde service neutron] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:19:33.129948 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44093815-890b-4cc4-b543-4ff25ee4fa5f admin admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:19:33.129948 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44093815-890b-4cc4-b543-4ff25ee4fa5f admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:19:33.129948 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44093815-890b-4cc4-b543-4ff25ee4fa5f admin admin] PATH_INFO: `/v3/domains` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:19:33.129948 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44093815-890b-4cc4-b543-4ff25ee4fa5f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:19:33.129948 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44093815-890b-4cc4-b543-4ff25ee4fa5f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:19:33.129948 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44093815-890b-4cc4-b543-4ff25ee4fa5f admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:19:33.135601 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 985/1993] 10.222.0.23 () {60 vars in 1365 bytes} [Thu May 19 12:19:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 12:19:33.145077 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1009/1994] 10.222.0.23 () {58 vars in 1152 bytes} [Thu May 19 12:19:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 12:19:33.146260 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e143993-e5f4-465e-9543-a8a41dfd5bde service neutron] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:19:33.158353 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e143993-e5f4-465e-9543-a8a41dfd5bde service neutron] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:19:33.163166 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf84269-d121-4fa5-a190-6b636e9a5a42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:19:33.164094 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf84269-d121-4fa5-a190-6b636e9a5a42 admin admin] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:19:33.164094 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf84269-d121-4fa5-a190-6b636e9a5a42 admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:19:33.164677 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf84269-d121-4fa5-a190-6b636e9a5a42 admin admin] PATH_INFO: `/v3/projects` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:19:33.165319 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf84269-d121-4fa5-a190-6b636e9a5a42 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:19:33.167260 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf84269-d121-4fa5-a190-6b636e9a5a42 admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:19:33.276077 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 986/1995] 10.222.0.23 () {60 vars in 1152 bytes} [Thu May 19 12:19:33 2022] POST /identity/v3/projects => generated 383 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 12:19:33.276874 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44093815-890b-4cc4-b543-4ff25ee4fa5f admin admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:19:33.287356 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44093815-890b-4cc4-b543-4ff25ee4fa5f admin admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:19:33.355978 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23b6617-6fa8-49cd-9bbb-ddb72cdb47ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:19:33.356671 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23b6617-6fa8-49cd-9bbb-ddb72cdb47ff admin admin] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:19:33.356964 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23b6617-6fa8-49cd-9bbb-ddb72cdb47ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:19:33.357247 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23b6617-6fa8-49cd-9bbb-ddb72cdb47ff admin admin] PATH_INFO: `/v3/users` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:19:33.358210 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23b6617-6fa8-49cd-9bbb-ddb72cdb47ff admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:19:33.360193 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23b6617-6fa8-49cd-9bbb-ddb72cdb47ff admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:19:33.457687 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1010/1996] 10.222.0.23 () {60 vars in 1143 bytes} [Thu May 19 12:19:33 2022] POST /identity/v3/users => generated 292 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 12:19:33.462176 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf84269-d121-4fa5-a190-6b636e9a5a42 admin admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:19:33.477268 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf84269-d121-4fa5-a190-6b636e9a5a42 admin admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:19:33.484116 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6e19e2-50ee-4b4a-88e9-cd46027f0657 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:19:33.484893 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6e19e2-50ee-4b4a-88e9-cd46027f0657 admin admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:19:33.485252 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6e19e2-50ee-4b4a-88e9-cd46027f0657 admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:19:33.485583 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6e19e2-50ee-4b4a-88e9-cd46027f0657 admin admin] PATH_INFO: `/v3/roles` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:19:33.486473 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6e19e2-50ee-4b4a-88e9-cd46027f0657 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:19:33.488362 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6e19e2-50ee-4b4a-88e9-cd46027f0657 admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:19:33.501389 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 987/1997] 10.222.0.23 () {58 vars in 1121 bytes} [Thu May 19 12:19:33 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 12:19:33.505484 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23b6617-6fa8-49cd-9bbb-ddb72cdb47ff admin admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:19:33.518609 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23b6617-6fa8-49cd-9bbb-ddb72cdb47ff admin admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:19:33.526084 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d76680-8a4e-47a5-b650-aee67c55e210 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:19:33.526839 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d76680-8a4e-47a5-b650-aee67c55e210 admin admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:19:33.527127 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d76680-8a4e-47a5-b650-aee67c55e210 admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:19:33.527405 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d76680-8a4e-47a5-b650-aee67c55e210 admin admin] PATH_INFO: `/v3/roles` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:19:33.528200 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d76680-8a4e-47a5-b650-aee67c55e210 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:19:33.529873 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d76680-8a4e-47a5-b650-aee67c55e210 admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:19:33.541835 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1011/1998] 10.222.0.23 () {58 vars in 1121 bytes} [Thu May 19 12:19:33 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 19 12:19:33.546454 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6e19e2-50ee-4b4a-88e9-cd46027f0657 admin admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:19:33.563068 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6e19e2-50ee-4b4a-88e9-cd46027f0657 admin admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:19:33.569141 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f6dfa4-c1df-4d46-b681-171f02563c03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:19:33.569141 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f6dfa4-c1df-4d46-b681-171f02563c03 admin admin] REQUEST_METHOD: `PUT` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:19:33.569466 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f6dfa4-c1df-4d46-b681-171f02563c03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:19:33.569627 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f6dfa4-c1df-4d46-b681-171f02563c03 admin admin] PATH_INFO: `/v3/projects/d47e754118cf4b08aa4ebdc4348f3b9b/users/2b217e73e3c34c97bce096f132960489/roles/259eb400cdb4439db72d14456ae9cd08` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:19:33.589343 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f6dfa4-c1df-4d46-b681-171f02563c03 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d47e754118cf4b08aa4ebdc4348f3b9b, user_id=2b217e73e3c34c97bce096f132960489, role_id=259eb400cdb4439db72d14456ae9cd08)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:19:33.592483 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f6dfa4-c1df-4d46-b681-171f02563c03 admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:19:33.702276 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 988/1999] 10.222.0.23 () {60 vars in 1482 bytes} [Thu May 19 12:19:33 2022] PUT /identity/v3/projects/d47e754118cf4b08aa4ebdc4348f3b9b/users/2b217e73e3c34c97bce096f132960489/roles/259eb400cdb4439db72d14456ae9cd08 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:19:33.719095 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5039b77a-5af6-40c0-a54c-1a5db7d7bb66 None None] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:19:33.719484 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5039b77a-5af6-40c0-a54c-1a5db7d7bb66 None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:19:33.719766 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5039b77a-5af6-40c0-a54c-1a5db7d7bb66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:19:33.751537 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.auth.core [None req-5039b77a-5af6-40c0-a54c-1a5db7d7bb66 None None] MFA Rules not processed for user `2b217e73e3c34c97bce096f132960489`. Rule list: `[]` (Enabled: `True`). {{(pid=80540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:19:33.753315 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-5039b77a-5af6-40c0-a54c-1a5db7d7bb66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:19:33.785887 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=93,keystone:DELETE=3,keystone:INSERT=4 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:19:33.837051 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1012/2000] 10.222.0.23 () {58 vars in 957 bytes} [Thu May 19 12:19:33 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:19:33.841720 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f6dfa4-c1df-4d46-b681-171f02563c03 admin admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:19:33.911628 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f6dfa4-c1df-4d46-b681-171f02563c03 admin admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:19:33.918010 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36916aae-164e-4ace-a57d-2905ffd1d0b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:19:33.918695 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36916aae-164e-4ace-a57d-2905ffd1d0b0 admin admin] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:19:33.919076 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36916aae-164e-4ace-a57d-2905ffd1d0b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:19:33.919281 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36916aae-164e-4ace-a57d-2905ffd1d0b0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:19:33.920032 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36916aae-164e-4ace-a57d-2905ffd1d0b0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:19:33.921975 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36916aae-164e-4ace-a57d-2905ffd1d0b0 admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:19:34.025577 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 989/2001] 10.222.0.23 () {60 vars in 1152 bytes} [Thu May 19 12:19:33 2022] POST /identity/v3/projects => generated 381 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 12:19:34.026371 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5039b77a-5af6-40c0-a54c-1a5db7d7bb66 None None] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:19:34.040333 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5039b77a-5af6-40c0-a54c-1a5db7d7bb66 None None] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:19:34.111911 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e096f6-6969-4ea4-a692-d2e8493d8596 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:19:34.112607 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e096f6-6969-4ea4-a692-d2e8493d8596 admin admin] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:19:34.112921 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e096f6-6969-4ea4-a692-d2e8493d8596 admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:19:34.113255 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e096f6-6969-4ea4-a692-d2e8493d8596 admin admin] PATH_INFO: `/v3/users` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:19:34.113955 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e096f6-6969-4ea4-a692-d2e8493d8596 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:19:34.115993 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e096f6-6969-4ea4-a692-d2e8493d8596 admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:19:34.238959 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1013/2002] 10.222.0.23 () {60 vars in 1143 bytes} [Thu May 19 12:19:34 2022] POST /identity/v3/users => generated 297 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 12:19:34.240339 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36916aae-164e-4ace-a57d-2905ffd1d0b0 admin admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:19:34.253548 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36916aae-164e-4ace-a57d-2905ffd1d0b0 admin admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:19:34.259625 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fd9f4f-43f0-4ef6-af8b-56cd85901d38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:19:34.261763 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fd9f4f-43f0-4ef6-af8b-56cd85901d38 admin admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:19:34.262445 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fd9f4f-43f0-4ef6-af8b-56cd85901d38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:19:34.262620 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fd9f4f-43f0-4ef6-af8b-56cd85901d38 admin admin] PATH_INFO: `/v3/roles` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:19:34.263489 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5fd9f4f-43f0-4ef6-af8b-56cd85901d38 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:19:34.265319 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5fd9f4f-43f0-4ef6-af8b-56cd85901d38 admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:19:34.281946 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 990/2003] 10.222.0.23 () {58 vars in 1121 bytes} [Thu May 19 12:19:34 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 12:19:34.287360 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e096f6-6969-4ea4-a692-d2e8493d8596 admin admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:19:34.300904 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e096f6-6969-4ea4-a692-d2e8493d8596 admin admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:19:34.306333 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c53d0bc-35db-459b-b43c-3888033caeea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:19:34.307247 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c53d0bc-35db-459b-b43c-3888033caeea admin admin] REQUEST_METHOD: `PUT` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:19:34.307539 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c53d0bc-35db-459b-b43c-3888033caeea admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:19:34.307857 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c53d0bc-35db-459b-b43c-3888033caeea admin admin] PATH_INFO: `/v3/projects/911f4bd1a81649c0929e5402dd27e278/users/516052af0bcb43038fdc8d81a9b029e5/roles/9db5671c65d3414ea2682afa2ccf1403` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:19:34.325576 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c53d0bc-35db-459b-b43c-3888033caeea admin admin] RBAC: Authorizing `identity:create_grant(project_id=911f4bd1a81649c0929e5402dd27e278, user_id=516052af0bcb43038fdc8d81a9b029e5, role_id=9db5671c65d3414ea2682afa2ccf1403)` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:19:34.328547 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c53d0bc-35db-459b-b43c-3888033caeea admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:19:34.428383 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1014/2004] 10.222.0.23 () {60 vars in 1482 bytes} [Thu May 19 12:19:34 2022] PUT /identity/v3/projects/911f4bd1a81649c0929e5402dd27e278/users/516052af0bcb43038fdc8d81a9b029e5/roles/9db5671c65d3414ea2682afa2ccf1403 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:19:34.432471 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fd9f4f-43f0-4ef6-af8b-56cd85901d38 admin admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:19:34.442633 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fd9f4f-43f0-4ef6-af8b-56cd85901d38 admin admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:19:34.472957 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:19:34.507930 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c906a1-b1bd-4ce0-9469-d68682988b6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:19:34.507930 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c906a1-b1bd-4ce0-9469-d68682988b6a admin admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:19:34.507930 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c906a1-b1bd-4ce0-9469-d68682988b6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:19:34.507930 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c906a1-b1bd-4ce0-9469-d68682988b6a admin admin] PATH_INFO: `/v3/roles` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:19:34.508642 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c906a1-b1bd-4ce0-9469-d68682988b6a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:19:34.510317 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c906a1-b1bd-4ce0-9469-d68682988b6a admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:19:34.520859 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 991/2005] 10.222.0.23 () {58 vars in 1121 bytes} [Thu May 19 12:19:34 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 12:19:34.526485 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c53d0bc-35db-459b-b43c-3888033caeea admin admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:19:34.538083 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c53d0bc-35db-459b-b43c-3888033caeea admin admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:19:34.545469 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c1df21-1818-4d49-84fe-ab873b8a151b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:19:34.546609 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c1df21-1818-4d49-84fe-ab873b8a151b admin admin] REQUEST_METHOD: `PUT` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:19:34.552430 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c1df21-1818-4d49-84fe-ab873b8a151b admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:19:34.552430 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c1df21-1818-4d49-84fe-ab873b8a151b admin admin] PATH_INFO: `/v3/projects/911f4bd1a81649c0929e5402dd27e278/users/516052af0bcb43038fdc8d81a9b029e5/roles/9db5671c65d3414ea2682afa2ccf1403` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:19:34.552430 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c1df21-1818-4d49-84fe-ab873b8a151b admin admin] RBAC: Authorizing `identity:create_grant(project_id=911f4bd1a81649c0929e5402dd27e278, user_id=516052af0bcb43038fdc8d81a9b029e5, role_id=9db5671c65d3414ea2682afa2ccf1403)` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:19:34.552749 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c1df21-1818-4d49-84fe-ab873b8a151b admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:19:34.565474 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1015/2006] 10.222.0.23 () {60 vars in 1482 bytes} [Thu May 19 12:19:34 2022] PUT /identity/v3/projects/911f4bd1a81649c0929e5402dd27e278/users/516052af0bcb43038fdc8d81a9b029e5/roles/9db5671c65d3414ea2682afa2ccf1403 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:19:34.580624 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a60d4b-f988-425c-93f3-633e095957a2 None None] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:19:34.580624 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a60d4b-f988-425c-93f3-633e095957a2 None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:19:34.586109 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a60d4b-f988-425c-93f3-633e095957a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:19:34.610388 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.auth.core [None req-d3a60d4b-f988-425c-93f3-633e095957a2 None None] MFA Rules not processed for user `516052af0bcb43038fdc8d81a9b029e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:19:34.611603 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-d3a60d4b-f988-425c-93f3-633e095957a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:19:34.683425 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 992/2007] 10.222.0.23 () {58 vars in 957 bytes} [Thu May 19 12:19:34 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:19:34.696671 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c1df21-1818-4d49-84fe-ab873b8a151b admin admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:19:34.698387 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-99c1df21-1818-4d49-84fe-ab873b8a151b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:19:34.785001 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c1df21-1818-4d49-84fe-ab873b8a151b admin admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:19:34.800775 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c50d220-c8b0-4252-be03-3f440c421511 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8800254c0e784e42a791fb9abc274d79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd70d25bf69314820b8aaecb65576978a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:19:34.801478 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c50d220-c8b0-4252-be03-3f440c421511 service cinder] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:19:34.801763 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c50d220-c8b0-4252-be03-3f440c421511 service cinder] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:19:34.802038 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c50d220-c8b0-4252-be03-3f440c421511 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:19:34.812764 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c50d220-c8b0-4252-be03-3f440c421511 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:19:34.814219 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c50d220-c8b0-4252-be03-3f440c421511 service cinder] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:19:34.819619 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:19:34.825661 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1016/2008] 10.222.0.23 () {60 vars in 1365 bytes} [Thu May 19 12:19:34 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:19:34.910312 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2596c507-aa07-4bb4-b873-64c5cf51c20b None None] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:19:34.910688 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2596c507-aa07-4bb4-b873-64c5cf51c20b None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:19:34.911060 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2596c507-aa07-4bb4-b873-64c5cf51c20b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:19:34.943135 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.auth.core [None req-2596c507-aa07-4bb4-b873-64c5cf51c20b None None] MFA Rules not processed for user `858ef1572f064c248592e92625b228f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:19:34.945628 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-2596c507-aa07-4bb4-b873-64c5cf51c20b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:19:35.012724 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c50d220-c8b0-4252-be03-3f440c421511 service cinder] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:19:35.021646 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c50d220-c8b0-4252-be03-3f440c421511 service cinder] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:19:35.026144 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79707c0e-316c-474f-9575-6ff2c86d5294 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8800254c0e784e42a791fb9abc274d79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd70d25bf69314820b8aaecb65576978a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:19:35.027547 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79707c0e-316c-474f-9575-6ff2c86d5294 service cinder] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:19:35.027547 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79707c0e-316c-474f-9575-6ff2c86d5294 service cinder] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:19:35.027547 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79707c0e-316c-474f-9575-6ff2c86d5294 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:19:35.028975 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 993/2009] 10.222.0.23 () {58 vars in 957 bytes} [Thu May 19 12:19:34 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:19:35.040300 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2596c507-aa07-4bb4-b873-64c5cf51c20b None None] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:19:35.041704 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-2596c507-aa07-4bb4-b873-64c5cf51c20b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:19:35.053697 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2596c507-aa07-4bb4-b873-64c5cf51c20b None None] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:19:35.058147 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12aeb369-7c4a-4d85-ac0a-e574481cf89b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8800254c0e784e42a791fb9abc274d79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd70d25bf69314820b8aaecb65576978a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:19:35.058803 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12aeb369-7c4a-4d85-ac0a-e574481cf89b service cinder] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:19:35.059091 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12aeb369-7c4a-4d85-ac0a-e574481cf89b service cinder] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:19:35.059373 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12aeb369-7c4a-4d85-ac0a-e574481cf89b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:19:35.068829 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12aeb369-7c4a-4d85-ac0a-e574481cf89b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:19:35.070943 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12aeb369-7c4a-4d85-ac0a-e574481cf89b service cinder] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:19:35.080999 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 994/2010] 10.222.0.23 () {60 vars in 1365 bytes} [Thu May 19 12:19:35 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 12:19:35.094606 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79707c0e-316c-474f-9575-6ff2c86d5294 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:19:35.096186 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79707c0e-316c-474f-9575-6ff2c86d5294 service cinder] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:19:35.106696 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1017/2011] 10.222.0.23 () {60 vars in 1365 bytes} [Thu May 19 12:19:35 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:19:35.857103 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12aeb369-7c4a-4d85-ac0a-e574481cf89b service cinder] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:19:35.891191 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:19:35.928754 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12aeb369-7c4a-4d85-ac0a-e574481cf89b service cinder] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:19:35.934247 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d524658d-9e02-4c68-aae4-eea78b489166 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:19:35.935151 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d524658d-9e02-4c68-aae4-eea78b489166 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:19:35.935436 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d524658d-9e02-4c68-aae4-eea78b489166 admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:19:35.935715 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d524658d-9e02-4c68-aae4-eea78b489166 admin admin] PATH_INFO: `/v3/users/2b217e73e3c34c97bce096f132960489` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:19:35.937048 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d524658d-9e02-4c68-aae4-eea78b489166 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2b217e73e3c34c97bce096f132960489)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:19:35.938998 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d524658d-9e02-4c68-aae4-eea78b489166 admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:19:36.154421 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-d524658d-9e02-4c68-aae4-eea78b489166 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2b217e73e3c34c97bce096f132960489'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:19:36.170083 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-d524658d-9e02-4c68-aae4-eea78b489166 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2b217e73e3c34c97bce096f132960489'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:19:36.171023 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-d524658d-9e02-4c68-aae4-eea78b489166 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2b217e73e3c34c97bce096f132960489'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:19:36.285918 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-d524658d-9e02-4c68-aae4-eea78b489166 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2b217e73e3c34c97bce096f132960489'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:19:36.295290 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-d524658d-9e02-4c68-aae4-eea78b489166 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2b217e73e3c34c97bce096f132960489'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:19:36.302475 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 995/2012] 10.222.0.23 () {58 vars in 1223 bytes} [Thu May 19 12:19:35 2022] DELETE /identity/v3/users/2b217e73e3c34c97bce096f132960489 => generated 0 bytes in 446 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:19:36.307214 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e03138-7a40-40b6-997a-13e8a3b6e68e None None] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:19:36.307729 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e03138-7a40-40b6-997a-13e8a3b6e68e None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:19:36.307842 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e03138-7a40-40b6-997a-13e8a3b6e68e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:19:36.326562 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.auth.core [None req-a2e03138-7a40-40b6-997a-13e8a3b6e68e None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:19:36.327830 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-a2e03138-7a40-40b6-997a-13e8a3b6e68e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:19:36.390192 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1018/2013] 10.222.0.23 () {58 vars in 957 bytes} [Thu May 19 12:19:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 12:19:36.400368 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d524658d-9e02-4c68-aae4-eea78b489166 admin admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:19:36.401816 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-d524658d-9e02-4c68-aae4-eea78b489166 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:19:36.463679 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d524658d-9e02-4c68-aae4-eea78b489166 admin admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:19:36.467440 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47fda49-05ad-4a3b-9ffc-dcd7b182d66e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a00b3fa7497456895e5e0bf105c4fc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd70d25bf69314820b8aaecb65576978a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:19:36.468119 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47fda49-05ad-4a3b-9ffc-dcd7b182d66e service neutron] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:19:36.468403 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47fda49-05ad-4a3b-9ffc-dcd7b182d66e service neutron] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:19:36.468677 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47fda49-05ad-4a3b-9ffc-dcd7b182d66e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:19:36.480109 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47fda49-05ad-4a3b-9ffc-dcd7b182d66e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:19:36.481577 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47fda49-05ad-4a3b-9ffc-dcd7b182d66e service neutron] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:19:36.493025 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 996/2014] 10.222.0.23 () {60 vars in 1365 bytes} [Thu May 19 12:19:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:19:37.172935 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e03138-7a40-40b6-997a-13e8a3b6e68e None None] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:19:37.174508 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-a2e03138-7a40-40b6-997a-13e8a3b6e68e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:19:37.187957 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e03138-7a40-40b6-997a-13e8a3b6e68e None None] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:19:37.192920 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47227e4-2f86-4098-bad5-4565d34be6cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:19:37.193871 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47227e4-2f86-4098-bad5-4565d34be6cd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:19:37.194162 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47227e4-2f86-4098-bad5-4565d34be6cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:19:37.194443 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47227e4-2f86-4098-bad5-4565d34be6cd admin admin] PATH_INFO: `/v3/projects/d47e754118cf4b08aa4ebdc4348f3b9b` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:19:37.197065 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c47227e4-2f86-4098-bad5-4565d34be6cd admin admin] RBAC: Authorizing `identity:delete_project(project_id=d47e754118cf4b08aa4ebdc4348f3b9b)` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:19:37.199010 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c47227e4-2f86-4098-bad5-4565d34be6cd admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:19:37.342661 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-c47227e4-2f86-4098-bad5-4565d34be6cd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd47e754118cf4b08aa4ebdc4348f3b9b', 'request_id': 'req-c47227e4-2f86-4098-bad5-4565d34be6cd'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:19:37.349671 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-c47227e4-2f86-4098-bad5-4565d34be6cd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd47e754118cf4b08aa4ebdc4348f3b9b', 'request_id': 'req-c47227e4-2f86-4098-bad5-4565d34be6cd'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:19:37.356431 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-c47227e4-2f86-4098-bad5-4565d34be6cd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd47e754118cf4b08aa4ebdc4348f3b9b', 'request_id': 'req-c47227e4-2f86-4098-bad5-4565d34be6cd'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:19:37.459281 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-c47227e4-2f86-4098-bad5-4565d34be6cd admin admin] The token cache is being invalidate because project d47e754118cf4b08aa4ebdc4348f3b9b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80540) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 12:19:37.464666 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-c47227e4-2f86-4098-bad5-4565d34be6cd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:19:37.464666 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-c47227e4-2f86-4098-bad5-4565d34be6cd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:19:37.464666 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1019/2015] 10.222.0.23 () {58 vars in 1232 bytes} [Thu May 19 12:19:37 2022] DELETE /identity/v3/projects/d47e754118cf4b08aa4ebdc4348f3b9b => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:19:37.466615 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47fda49-05ad-4a3b-9ffc-dcd7b182d66e service neutron] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:19:37.479085 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47fda49-05ad-4a3b-9ffc-dcd7b182d66e service neutron] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:19:37.536698 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:19:37.549599 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab84f4c3-cbf6-4b71-aed6-75a9c08018a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:19:37.550496 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab84f4c3-cbf6-4b71-aed6-75a9c08018a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:19:37.550944 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab84f4c3-cbf6-4b71-aed6-75a9c08018a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:19:37.551148 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab84f4c3-cbf6-4b71-aed6-75a9c08018a9 admin admin] PATH_INFO: `/v3/users/516052af0bcb43038fdc8d81a9b029e5` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:19:37.552417 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab84f4c3-cbf6-4b71-aed6-75a9c08018a9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=516052af0bcb43038fdc8d81a9b029e5)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:19:37.555302 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab84f4c3-cbf6-4b71-aed6-75a9c08018a9 admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:19:37.798694 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-ab84f4c3-cbf6-4b71-aed6-75a9c08018a9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '516052af0bcb43038fdc8d81a9b029e5'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:19:37.811268 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-ab84f4c3-cbf6-4b71-aed6-75a9c08018a9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '516052af0bcb43038fdc8d81a9b029e5'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:19:37.812025 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-ab84f4c3-cbf6-4b71-aed6-75a9c08018a9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '516052af0bcb43038fdc8d81a9b029e5'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:19:37.902315 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-ab84f4c3-cbf6-4b71-aed6-75a9c08018a9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '516052af0bcb43038fdc8d81a9b029e5'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:19:37.909898 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-ab84f4c3-cbf6-4b71-aed6-75a9c08018a9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '516052af0bcb43038fdc8d81a9b029e5'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:19:37.917771 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 997/2016] 10.222.0.23 () {58 vars in 1223 bytes} [Thu May 19 12:19:37 2022] DELETE /identity/v3/users/516052af0bcb43038fdc8d81a9b029e5 => generated 0 bytes in 452 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:19:38.057051 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cd0472-9450-4f70-a579-db4bef44b5a6 None None] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:19:38.057335 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cd0472-9450-4f70-a579-db4bef44b5a6 None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:19:38.057621 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cd0472-9450-4f70-a579-db4bef44b5a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:19:38.077103 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.auth.core [None req-27cd0472-9450-4f70-a579-db4bef44b5a6 None None] MFA Rules not processed for user `858ef1572f064c248592e92625b228f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:19:38.078322 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-27cd0472-9450-4f70-a579-db4bef44b5a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:19:38.096468 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:19:38.165645 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1020/2017] 10.222.0.23 () {58 vars in 957 bytes} [Thu May 19 12:19:38 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:19:38.612085 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab84f4c3-cbf6-4b71-aed6-75a9c08018a9 admin admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:19:38.613895 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-ab84f4c3-cbf6-4b71-aed6-75a9c08018a9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:19:38.742269 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab84f4c3-cbf6-4b71-aed6-75a9c08018a9 admin admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:19:38.748687 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92e3a68-19c6-4e98-9f4e-48e08abab8b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:19:38.749823 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92e3a68-19c6-4e98-9f4e-48e08abab8b5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:19:38.750221 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92e3a68-19c6-4e98-9f4e-48e08abab8b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:19:38.750684 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92e3a68-19c6-4e98-9f4e-48e08abab8b5 admin admin] PATH_INFO: `/v3/projects/911f4bd1a81649c0929e5402dd27e278` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:19:38.752746 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92e3a68-19c6-4e98-9f4e-48e08abab8b5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=911f4bd1a81649c0929e5402dd27e278)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:19:38.755187 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92e3a68-19c6-4e98-9f4e-48e08abab8b5 admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:19:38.959293 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-b92e3a68-19c6-4e98-9f4e-48e08abab8b5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '911f4bd1a81649c0929e5402dd27e278', 'request_id': 'req-b92e3a68-19c6-4e98-9f4e-48e08abab8b5'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:19:38.975443 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-b92e3a68-19c6-4e98-9f4e-48e08abab8b5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '911f4bd1a81649c0929e5402dd27e278', 'request_id': 'req-b92e3a68-19c6-4e98-9f4e-48e08abab8b5'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:19:39.172422 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-b92e3a68-19c6-4e98-9f4e-48e08abab8b5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '911f4bd1a81649c0929e5402dd27e278', 'request_id': 'req-b92e3a68-19c6-4e98-9f4e-48e08abab8b5'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:19:39.182421 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:19:39.189602 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-b92e3a68-19c6-4e98-9f4e-48e08abab8b5 admin admin] The token cache is being invalidate because project 911f4bd1a81649c0929e5402dd27e278 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80539) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 12:19:39.190074 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-b92e3a68-19c6-4e98-9f4e-48e08abab8b5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:19:39.191554 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-b92e3a68-19c6-4e98-9f4e-48e08abab8b5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:19:39.202409 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 998/2018] 10.222.0.23 () {58 vars in 1232 bytes} [Thu May 19 12:19:38 2022] DELETE /identity/v3/projects/911f4bd1a81649c0929e5402dd27e278 => generated 0 bytes in 586 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:19:39.203233 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de142f4-415a-49cc-8a68-3de2ecc54c9c None None] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:19:39.203835 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de142f4-415a-49cc-8a68-3de2ecc54c9c None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:19:39.203921 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de142f4-415a-49cc-8a68-3de2ecc54c9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:19:39.226644 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.auth.core [None req-7de142f4-415a-49cc-8a68-3de2ecc54c9c None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:19:39.232557 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-7de142f4-415a-49cc-8a68-3de2ecc54c9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:19:39.338616 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1021/2019] 10.222.0.23 () {58 vars in 957 bytes} [Thu May 19 12:19:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:19:39.346549 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ca0d15-0df4-4490-be3c-214759547800 None None] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:19:39.346937 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ca0d15-0df4-4490-be3c-214759547800 None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:19:39.347134 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ca0d15-0df4-4490-be3c-214759547800 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:19:39.370733 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.auth.core [None req-71ca0d15-0df4-4490-be3c-214759547800 None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:19:39.372118 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-71ca0d15-0df4-4490-be3c-214759547800 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:19:39.381132 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 999/2020] 10.222.0.23 () {58 vars in 957 bytes} [Thu May 19 12:19:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 12:19:39.385815 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de142f4-415a-49cc-8a68-3de2ecc54c9c None None] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:19:39.401091 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de142f4-415a-49cc-8a68-3de2ecc54c9c None None] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:19:39.416116 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7493058-b992-4c36-b359-873f281c54a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:19:39.416116 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7493058-b992-4c36-b359-873f281c54a0 admin admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:19:39.416116 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7493058-b992-4c36-b359-873f281c54a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:19:39.416116 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7493058-b992-4c36-b359-873f281c54a0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:19:39.416458 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7493058-b992-4c36-b359-873f281c54a0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:19:39.416458 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7493058-b992-4c36-b359-873f281c54a0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:19:39.416458 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7493058-b992-4c36-b359-873f281c54a0 admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:19:39.428744 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1022/2021] 10.222.0.23 () {58 vars in 1152 bytes} [Thu May 19 12:19:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:19:39.433851 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ca0d15-0df4-4490-be3c-214759547800 None None] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:19:39.451202 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ca0d15-0df4-4490-be3c-214759547800 None None] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:19:39.456830 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dae8563-6ff0-4429-82f7-5eb1ca4076d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:19:39.457589 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dae8563-6ff0-4429-82f7-5eb1ca4076d6 admin admin] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:19:39.457942 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dae8563-6ff0-4429-82f7-5eb1ca4076d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:19:39.458292 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dae8563-6ff0-4429-82f7-5eb1ca4076d6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:19:39.459063 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dae8563-6ff0-4429-82f7-5eb1ca4076d6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:19:39.461335 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dae8563-6ff0-4429-82f7-5eb1ca4076d6 admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:19:39.509681 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7493058-b992-4c36-b359-873f281c54a0 admin admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:19:39.512024 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-e7493058-b992-4c36-b359-873f281c54a0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:19:39.528226 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7493058-b992-4c36-b359-873f281c54a0 admin admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:19:39.535680 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fcfc10a-4a93-4177-88af-ac798321a1dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:19:39.536679 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcfc10a-4a93-4177-88af-ac798321a1dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:19:39.536974 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcfc10a-4a93-4177-88af-ac798321a1dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:19:39.537256 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcfc10a-4a93-4177-88af-ac798321a1dc admin admin] PATH_INFO: `/v3/users/84344433142e473fb8bd12395c406a15` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:19:39.539069 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fcfc10a-4a93-4177-88af-ac798321a1dc admin admin] RBAC: Authorizing `identity:delete_user(user_id=84344433142e473fb8bd12395c406a15)` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:19:39.541999 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fcfc10a-4a93-4177-88af-ac798321a1dc admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:19:39.571113 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1000/2022] 10.222.0.23 () {60 vars in 1152 bytes} [Thu May 19 12:19:39 2022] POST /identity/v3/projects => generated 383 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 12:19:39.575890 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dae8563-6ff0-4429-82f7-5eb1ca4076d6 admin admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:19:39.650267 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dae8563-6ff0-4429-82f7-5eb1ca4076d6 admin admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:19:39.655433 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81c231e-a374-4018-9789-4c0524cdea75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:19:39.656224 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81c231e-a374-4018-9789-4c0524cdea75 admin admin] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:19:39.656574 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81c231e-a374-4018-9789-4c0524cdea75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:19:39.656859 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81c231e-a374-4018-9789-4c0524cdea75 admin admin] PATH_INFO: `/v3/users` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:19:39.657569 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81c231e-a374-4018-9789-4c0524cdea75 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:19:39.659717 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81c231e-a374-4018-9789-4c0524cdea75 admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:19:39.764538 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1001/2023] 10.222.0.23 () {60 vars in 1143 bytes} [Thu May 19 12:19:39 2022] POST /identity/v3/users => generated 292 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 12:19:39.773735 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-1fcfc10a-4a93-4177-88af-ac798321a1dc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '84344433142e473fb8bd12395c406a15'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:19:39.775157 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81c231e-a374-4018-9789-4c0524cdea75 admin admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:19:39.799358 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-1fcfc10a-4a93-4177-88af-ac798321a1dc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '84344433142e473fb8bd12395c406a15'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:19:39.799358 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-1fcfc10a-4a93-4177-88af-ac798321a1dc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '84344433142e473fb8bd12395c406a15'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:19:39.802605 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81c231e-a374-4018-9789-4c0524cdea75 admin admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:19:39.812175 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1315e92a-056d-49c1-8419-0eca00545b60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:19:39.812927 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1315e92a-056d-49c1-8419-0eca00545b60 admin admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:19:39.813210 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1315e92a-056d-49c1-8419-0eca00545b60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:19:39.813513 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1315e92a-056d-49c1-8419-0eca00545b60 admin admin] PATH_INFO: `/v3/roles` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:19:39.814295 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1315e92a-056d-49c1-8419-0eca00545b60 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:19:39.816094 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1315e92a-056d-49c1-8419-0eca00545b60 admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:19:39.827614 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1002/2024] 10.222.0.23 () {58 vars in 1121 bytes} [Thu May 19 12:19:39 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 12:19:39.831572 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1315e92a-056d-49c1-8419-0eca00545b60 admin admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:19:39.841775 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1315e92a-056d-49c1-8419-0eca00545b60 admin admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:19:39.846822 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354c51a6-d785-4288-b0f0-5b2030cf4b88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:19:39.847439 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354c51a6-d785-4288-b0f0-5b2030cf4b88 admin admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:19:39.847731 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354c51a6-d785-4288-b0f0-5b2030cf4b88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:19:39.848055 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354c51a6-d785-4288-b0f0-5b2030cf4b88 admin admin] PATH_INFO: `/v3/roles` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:19:39.848790 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354c51a6-d785-4288-b0f0-5b2030cf4b88 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:19:39.850564 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354c51a6-d785-4288-b0f0-5b2030cf4b88 admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:19:39.864134 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1003/2025] 10.222.0.23 () {58 vars in 1121 bytes} [Thu May 19 12:19:39 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 12:19:39.868836 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354c51a6-d785-4288-b0f0-5b2030cf4b88 admin admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:19:39.879314 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354c51a6-d785-4288-b0f0-5b2030cf4b88 admin admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:19:39.885107 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172282ed-46cc-492f-a95b-3bae9e1b6a7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:19:39.886270 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172282ed-46cc-492f-a95b-3bae9e1b6a7a admin admin] REQUEST_METHOD: `PUT` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:19:39.886391 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172282ed-46cc-492f-a95b-3bae9e1b6a7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:19:39.891587 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172282ed-46cc-492f-a95b-3bae9e1b6a7a admin admin] PATH_INFO: `/v3/projects/01164d76ccfa447ba19e004349921bfb/users/05053998a7fc4fe296698ebc085a68bf/roles/259eb400cdb4439db72d14456ae9cd08` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:19:39.891727 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-1fcfc10a-4a93-4177-88af-ac798321a1dc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '84344433142e473fb8bd12395c406a15'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:19:39.902910 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-1fcfc10a-4a93-4177-88af-ac798321a1dc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '84344433142e473fb8bd12395c406a15'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:19:39.910528 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1023/2026] 10.222.0.23 () {58 vars in 1223 bytes} [Thu May 19 12:19:39 2022] DELETE /identity/v3/users/84344433142e473fb8bd12395c406a15 => generated 0 bytes in 401 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 12:19:39.916117 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3dfbd65-9a09-4563-891d-251bb8be3897 None None] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:19:39.916532 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3dfbd65-9a09-4563-891d-251bb8be3897 None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:19:39.916823 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3dfbd65-9a09-4563-891d-251bb8be3897 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:19:39.917207 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172282ed-46cc-492f-a95b-3bae9e1b6a7a admin admin] RBAC: Authorizing `identity:create_grant(project_id=01164d76ccfa447ba19e004349921bfb, user_id=05053998a7fc4fe296698ebc085a68bf, role_id=259eb400cdb4439db72d14456ae9cd08)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:19:39.920061 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172282ed-46cc-492f-a95b-3bae9e1b6a7a admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:19:39.938601 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.auth.core [None req-b3dfbd65-9a09-4563-891d-251bb8be3897 None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:19:39.940120 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-b3dfbd65-9a09-4563-891d-251bb8be3897 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:19:39.964999 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:19:40.006853 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1024/2027] 10.222.0.23 () {58 vars in 957 bytes} [Thu May 19 12:19:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:19:40.018508 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3dfbd65-9a09-4563-891d-251bb8be3897 None None] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:19:40.021134 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-b3dfbd65-9a09-4563-891d-251bb8be3897 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:19:40.031396 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1004/2028] 10.222.0.23 () {60 vars in 1482 bytes} [Thu May 19 12:19:39 2022] PUT /identity/v3/projects/01164d76ccfa447ba19e004349921bfb/users/05053998a7fc4fe296698ebc085a68bf/roles/259eb400cdb4439db72d14456ae9cd08 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 12:19:40.048549 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be05ac1a-2b40-4580-8de9-7062c203ed66 None None] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:19:40.048869 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be05ac1a-2b40-4580-8de9-7062c203ed66 None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:19:40.049136 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be05ac1a-2b40-4580-8de9-7062c203ed66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:19:40.089105 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.auth.core [None req-be05ac1a-2b40-4580-8de9-7062c203ed66 None None] MFA Rules not processed for user `05053998a7fc4fe296698ebc085a68bf`. Rule list: `[]` (Enabled: `True`). {{(pid=80539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:19:40.091218 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-be05ac1a-2b40-4580-8de9-7062c203ed66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:19:40.098235 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3dfbd65-9a09-4563-891d-251bb8be3897 None None] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:19:40.174812 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ad53d0-b237-4db2-9b13-0543430a6f68 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a00b3fa7497456895e5e0bf105c4fc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd70d25bf69314820b8aaecb65576978a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:19:40.175465 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ad53d0-b237-4db2-9b13-0543430a6f68 service neutron] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:19:40.175777 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ad53d0-b237-4db2-9b13-0543430a6f68 service neutron] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:19:40.178777 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ad53d0-b237-4db2-9b13-0543430a6f68 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:19:40.214638 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1005/2029] 10.222.0.23 () {58 vars in 957 bytes} [Thu May 19 12:19:40 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:19:40.219079 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be05ac1a-2b40-4580-8de9-7062c203ed66 None None] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:19:40.220850 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-be05ac1a-2b40-4580-8de9-7062c203ed66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:19:40.229557 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:19:40.280338 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ad53d0-b237-4db2-9b13-0543430a6f68 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:19:40.282795 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ad53d0-b237-4db2-9b13-0543430a6f68 service neutron] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:19:40.300145 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1025/2030] 10.222.0.23 () {60 vars in 1365 bytes} [Thu May 19 12:19:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 282 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 12:19:40.383831 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be05ac1a-2b40-4580-8de9-7062c203ed66 None None] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:19:40.393949 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8594e00f-faba-48dc-a3dd-7c995c1db1d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:19:40.407732 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8594e00f-faba-48dc-a3dd-7c995c1db1d7 admin admin] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:19:40.408706 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8594e00f-faba-48dc-a3dd-7c995c1db1d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:19:40.409098 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8594e00f-faba-48dc-a3dd-7c995c1db1d7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:19:40.409958 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8594e00f-faba-48dc-a3dd-7c995c1db1d7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:19:40.428915 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8594e00f-faba-48dc-a3dd-7c995c1db1d7 admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:19:40.548250 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ad53d0-b237-4db2-9b13-0543430a6f68 service neutron] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:19:40.549722 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1006/2031] 10.222.0.23 () {60 vars in 1152 bytes} [Thu May 19 12:19:40 2022] POST /identity/v3/projects => generated 385 bytes in 331 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 12:19:40.567428 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ad53d0-b237-4db2-9b13-0543430a6f68 service neutron] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:19:40.610436 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:19:40.648335 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53b45ec-708d-4208-a03e-9f37600b5071 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:19:40.649190 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53b45ec-708d-4208-a03e-9f37600b5071 admin admin] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:19:40.649794 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53b45ec-708d-4208-a03e-9f37600b5071 admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:19:40.650167 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53b45ec-708d-4208-a03e-9f37600b5071 admin admin] PATH_INFO: `/v3/users` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:19:40.650886 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53b45ec-708d-4208-a03e-9f37600b5071 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:19:40.653008 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53b45ec-708d-4208-a03e-9f37600b5071 admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:19:40.780437 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1026/2032] 10.222.0.23 () {60 vars in 1143 bytes} [Thu May 19 12:19:40 2022] POST /identity/v3/users => generated 293 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 12:19:40.784817 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8594e00f-faba-48dc-a3dd-7c995c1db1d7 admin admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:19:40.797461 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8594e00f-faba-48dc-a3dd-7c995c1db1d7 admin admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:19:40.812110 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94eacb61-55b1-4361-a797-535e03b3efa9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:19:40.812110 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94eacb61-55b1-4361-a797-535e03b3efa9 admin admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:19:40.812110 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94eacb61-55b1-4361-a797-535e03b3efa9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:19:40.812110 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94eacb61-55b1-4361-a797-535e03b3efa9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:19:40.812468 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94eacb61-55b1-4361-a797-535e03b3efa9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:19:40.812468 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94eacb61-55b1-4361-a797-535e03b3efa9 admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:19:40.813844 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53b45ec-708d-4208-a03e-9f37600b5071 admin admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:19:40.819176 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-c53b45ec-708d-4208-a03e-9f37600b5071 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:19:40.835163 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53b45ec-708d-4208-a03e-9f37600b5071 admin admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:19:40.840342 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504bce22-7ebc-45cf-88c4-b5312fb94935 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:19:40.841366 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504bce22-7ebc-45cf-88c4-b5312fb94935 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:19:40.841744 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504bce22-7ebc-45cf-88c4-b5312fb94935 admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:19:40.842271 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504bce22-7ebc-45cf-88c4-b5312fb94935 admin admin] PATH_INFO: `/v3/projects/552bcbb8eaa74974a2fc2535c7b2d75a` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:19:40.843604 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1007/2033] 10.222.0.23 () {58 vars in 1121 bytes} [Thu May 19 12:19:40 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 12:19:40.847974 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504bce22-7ebc-45cf-88c4-b5312fb94935 admin admin] RBAC: Authorizing `identity:delete_project(project_id=552bcbb8eaa74974a2fc2535c7b2d75a)` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:19:40.848378 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94eacb61-55b1-4361-a797-535e03b3efa9 admin admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:19:40.854086 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504bce22-7ebc-45cf-88c4-b5312fb94935 admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:19:40.866103 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94eacb61-55b1-4361-a797-535e03b3efa9 admin admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:19:40.873287 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54200029-abbb-4a34-b10b-ea58942ef5a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:19:40.874216 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54200029-abbb-4a34-b10b-ea58942ef5a1 admin admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:19:40.874508 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54200029-abbb-4a34-b10b-ea58942ef5a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:19:40.874939 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54200029-abbb-4a34-b10b-ea58942ef5a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:19:40.875669 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54200029-abbb-4a34-b10b-ea58942ef5a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:19:40.878419 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54200029-abbb-4a34-b10b-ea58942ef5a1 admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:19:40.900780 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1008/2034] 10.222.0.23 () {58 vars in 1121 bytes} [Thu May 19 12:19:40 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 19 12:19:40.906596 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54200029-abbb-4a34-b10b-ea58942ef5a1 admin admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:19:40.952448 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54200029-abbb-4a34-b10b-ea58942ef5a1 admin admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:19:40.958337 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65608073-5894-4141-835f-d4a0ea993dfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:19:40.959979 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65608073-5894-4141-835f-d4a0ea993dfe admin admin] REQUEST_METHOD: `PUT` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:19:40.960328 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65608073-5894-4141-835f-d4a0ea993dfe admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:19:40.960690 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65608073-5894-4141-835f-d4a0ea993dfe admin admin] PATH_INFO: `/v3/projects/c7356f68a0aa43cfade192f26082f619/users/cbf69910293445a6b091ce90b1df1709/roles/259eb400cdb4439db72d14456ae9cd08` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:19:40.982937 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65608073-5894-4141-835f-d4a0ea993dfe admin admin] RBAC: Authorizing `identity:create_grant(project_id=c7356f68a0aa43cfade192f26082f619, user_id=cbf69910293445a6b091ce90b1df1709, role_id=259eb400cdb4439db72d14456ae9cd08)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:19:40.987177 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65608073-5894-4141-835f-d4a0ea993dfe admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:19:41.130384 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1009/2035] 10.222.0.23 () {60 vars in 1482 bytes} [Thu May 19 12:19:40 2022] PUT /identity/v3/projects/c7356f68a0aa43cfade192f26082f619/users/cbf69910293445a6b091ce90b1df1709/roles/259eb400cdb4439db72d14456ae9cd08 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:19:41.131954 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb690c7-b242-4688-86c2-57e241c1f3ff None None] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:19:41.132364 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb690c7-b242-4688-86c2-57e241c1f3ff None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:19:41.133175 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb690c7-b242-4688-86c2-57e241c1f3ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:19:41.138255 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-504bce22-7ebc-45cf-88c4-b5312fb94935 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '552bcbb8eaa74974a2fc2535c7b2d75a', 'request_id': 'req-504bce22-7ebc-45cf-88c4-b5312fb94935'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:19:41.149455 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-504bce22-7ebc-45cf-88c4-b5312fb94935 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '552bcbb8eaa74974a2fc2535c7b2d75a', 'request_id': 'req-504bce22-7ebc-45cf-88c4-b5312fb94935'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:19:41.161397 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-504bce22-7ebc-45cf-88c4-b5312fb94935 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '552bcbb8eaa74974a2fc2535c7b2d75a', 'request_id': 'req-504bce22-7ebc-45cf-88c4-b5312fb94935'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:19:41.168938 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.auth.core [None req-bcb690c7-b242-4688-86c2-57e241c1f3ff None None] MFA Rules not processed for user `cbf69910293445a6b091ce90b1df1709`. Rule list: `[]` (Enabled: `True`). {{(pid=80539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:19:41.170589 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-bcb690c7-b242-4688-86c2-57e241c1f3ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:19:41.263393 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1010/2036] 10.222.0.23 () {58 vars in 957 bytes} [Thu May 19 12:19:41 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:19:41.268749 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb690c7-b242-4688-86c2-57e241c1f3ff None None] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:19:41.316252 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:19:41.358620 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-504bce22-7ebc-45cf-88c4-b5312fb94935 admin admin] The token cache is being invalidate because project 552bcbb8eaa74974a2fc2535c7b2d75a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80540) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 12:19:41.359548 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb690c7-b242-4688-86c2-57e241c1f3ff None None] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:19:41.360680 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-504bce22-7ebc-45cf-88c4-b5312fb94935 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:19:41.362552 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-504bce22-7ebc-45cf-88c4-b5312fb94935 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:19:41.369003 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf45ed0-4596-4853-a05a-d1c8c9e6541f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:19:41.369821 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf45ed0-4596-4853-a05a-d1c8c9e6541f admin admin] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:19:41.370111 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf45ed0-4596-4853-a05a-d1c8c9e6541f admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:19:41.370270 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1027/2037] 10.222.0.23 () {58 vars in 1232 bytes} [Thu May 19 12:19:40 2022] DELETE /identity/v3/projects/552bcbb8eaa74974a2fc2535c7b2d75a => generated 0 bytes in 557 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:19:41.370395 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf45ed0-4596-4853-a05a-d1c8c9e6541f admin admin] PATH_INFO: `/v3/projects` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:19:41.371500 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf45ed0-4596-4853-a05a-d1c8c9e6541f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:19:41.373792 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf45ed0-4596-4853-a05a-d1c8c9e6541f admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:19:41.377310 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504bce22-7ebc-45cf-88c4-b5312fb94935 admin admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:19:41.379952 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-504bce22-7ebc-45cf-88c4-b5312fb94935 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:19:41.402711 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504bce22-7ebc-45cf-88c4-b5312fb94935 admin admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:19:41.407729 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2731bc-3645-4c5e-8186-3f9dba3268e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:19:41.408688 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2731bc-3645-4c5e-8186-3f9dba3268e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:19:41.409175 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2731bc-3645-4c5e-8186-3f9dba3268e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:19:41.409465 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2731bc-3645-4c5e-8186-3f9dba3268e6 admin admin] PATH_INFO: `/v3/users/805c0e51d9434a5aa509f390865569a6` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:19:41.410864 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2731bc-3645-4c5e-8186-3f9dba3268e6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=805c0e51d9434a5aa509f390865569a6)` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:19:41.414605 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2731bc-3645-4c5e-8186-3f9dba3268e6 admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:19:41.571821 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1011/2038] 10.222.0.23 () {60 vars in 1152 bytes} [Thu May 19 12:19:41 2022] POST /identity/v3/projects => generated 381 bytes in 301 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 12:19:41.573751 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf45ed0-4596-4853-a05a-d1c8c9e6541f admin admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:19:41.576161 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-daf45ed0-4596-4853-a05a-d1c8c9e6541f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:19:41.678127 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf45ed0-4596-4853-a05a-d1c8c9e6541f admin admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:19:41.681902 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ef1956-ca06-49da-a3c6-acedc0a47603 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:19:41.682596 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ef1956-ca06-49da-a3c6-acedc0a47603 admin admin] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:19:41.682978 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ef1956-ca06-49da-a3c6-acedc0a47603 admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:19:41.683219 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ef1956-ca06-49da-a3c6-acedc0a47603 admin admin] PATH_INFO: `/v3/users` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:19:41.683931 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ef1956-ca06-49da-a3c6-acedc0a47603 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:19:41.685922 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ef1956-ca06-49da-a3c6-acedc0a47603 admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:19:41.815030 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-3e2731bc-3645-4c5e-8186-3f9dba3268e6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '805c0e51d9434a5aa509f390865569a6'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:19:41.828520 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-3e2731bc-3645-4c5e-8186-3f9dba3268e6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '805c0e51d9434a5aa509f390865569a6'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:19:41.829336 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-3e2731bc-3645-4c5e-8186-3f9dba3268e6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '805c0e51d9434a5aa509f390865569a6'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:19:41.841402 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1012/2039] 10.222.0.23 () {60 vars in 1143 bytes} [Thu May 19 12:19:41 2022] POST /identity/v3/users => generated 297 bytes in 269 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 12:19:41.845462 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ef1956-ca06-49da-a3c6-acedc0a47603 admin admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:19:41.855690 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ef1956-ca06-49da-a3c6-acedc0a47603 admin admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:19:41.860498 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b0dcd0-eb62-4b20-94cc-c478f2d74950 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:19:41.861143 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b0dcd0-eb62-4b20-94cc-c478f2d74950 admin admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:19:41.861445 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b0dcd0-eb62-4b20-94cc-c478f2d74950 admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:19:41.861738 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b0dcd0-eb62-4b20-94cc-c478f2d74950 admin admin] PATH_INFO: `/v3/roles` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:19:41.862484 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b0dcd0-eb62-4b20-94cc-c478f2d74950 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:19:41.864166 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b0dcd0-eb62-4b20-94cc-c478f2d74950 admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:19:41.875432 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1013/2040] 10.222.0.23 () {58 vars in 1121 bytes} [Thu May 19 12:19:41 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 19 12:19:41.886269 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b0dcd0-eb62-4b20-94cc-c478f2d74950 admin admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:19:41.896224 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b0dcd0-eb62-4b20-94cc-c478f2d74950 admin admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:19:41.901363 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34da1bf2-f8df-4c23-bf6d-ddb923e9f0ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:19:41.902245 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34da1bf2-f8df-4c23-bf6d-ddb923e9f0ed admin admin] REQUEST_METHOD: `PUT` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:19:41.902529 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34da1bf2-f8df-4c23-bf6d-ddb923e9f0ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:19:41.902810 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34da1bf2-f8df-4c23-bf6d-ddb923e9f0ed admin admin] PATH_INFO: `/v3/projects/adc7c7b781734864a8f2977b38b85479/users/ac09aa384cd8426ba338b618e765c1ca/roles/9db5671c65d3414ea2682afa2ccf1403` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:19:41.920048 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34da1bf2-f8df-4c23-bf6d-ddb923e9f0ed admin admin] RBAC: Authorizing `identity:create_grant(project_id=adc7c7b781734864a8f2977b38b85479, user_id=ac09aa384cd8426ba338b618e765c1ca, role_id=9db5671c65d3414ea2682afa2ccf1403)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:19:41.922740 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34da1bf2-f8df-4c23-bf6d-ddb923e9f0ed admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:19:41.947881 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-3e2731bc-3645-4c5e-8186-3f9dba3268e6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '805c0e51d9434a5aa509f390865569a6'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:19:41.956082 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-3e2731bc-3645-4c5e-8186-3f9dba3268e6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '805c0e51d9434a5aa509f390865569a6'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:19:41.963095 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1028/2041] 10.222.0.23 () {58 vars in 1223 bytes} [Thu May 19 12:19:41 2022] DELETE /identity/v3/users/805c0e51d9434a5aa509f390865569a6 => generated 0 bytes in 586 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:19:42.052173 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1014/2042] 10.222.0.23 () {60 vars in 1482 bytes} [Thu May 19 12:19:41 2022] PUT /identity/v3/projects/adc7c7b781734864a8f2977b38b85479/users/ac09aa384cd8426ba338b618e765c1ca/roles/9db5671c65d3414ea2682afa2ccf1403 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 12:19:42.057066 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2731bc-3645-4c5e-8186-3f9dba3268e6 admin admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:19:42.058740 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-3e2731bc-3645-4c5e-8186-3f9dba3268e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:19:42.081111 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=78,keystone:INSERT=5,keystone:DELETE=17 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:19:42.128205 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2731bc-3645-4c5e-8186-3f9dba3268e6 admin admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:19:42.192315 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114f14f0-a6f6-499e-8944-b44ce824bd15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:19:42.193005 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114f14f0-a6f6-499e-8944-b44ce824bd15 admin admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:19:42.193348 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114f14f0-a6f6-499e-8944-b44ce824bd15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:19:42.193790 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114f14f0-a6f6-499e-8944-b44ce824bd15 admin admin] PATH_INFO: `/v3/roles` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:19:42.194543 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114f14f0-a6f6-499e-8944-b44ce824bd15 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:19:42.196348 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114f14f0-a6f6-499e-8944-b44ce824bd15 admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:19:42.206974 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1029/2043] 10.222.0.23 () {58 vars in 1121 bytes} [Thu May 19 12:19:42 2022] GET /identity/v3/roles => generated 1420 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 12:19:42.211234 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34da1bf2-f8df-4c23-bf6d-ddb923e9f0ed admin admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:19:42.222398 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34da1bf2-f8df-4c23-bf6d-ddb923e9f0ed admin admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:19:42.228285 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ad7ae3-5bfc-4f73-a00d-0b324796e6fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:19:42.229460 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ad7ae3-5bfc-4f73-a00d-0b324796e6fe admin admin] REQUEST_METHOD: `PUT` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:19:42.229764 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ad7ae3-5bfc-4f73-a00d-0b324796e6fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:19:42.230042 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ad7ae3-5bfc-4f73-a00d-0b324796e6fe admin admin] PATH_INFO: `/v3/projects/adc7c7b781734864a8f2977b38b85479/users/ac09aa384cd8426ba338b618e765c1ca/roles/9db5671c65d3414ea2682afa2ccf1403` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:19:42.232049 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ad7ae3-5bfc-4f73-a00d-0b324796e6fe admin admin] RBAC: Authorizing `identity:create_grant(project_id=adc7c7b781734864a8f2977b38b85479, user_id=ac09aa384cd8426ba338b618e765c1ca, role_id=9db5671c65d3414ea2682afa2ccf1403)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:19:42.234810 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ad7ae3-5bfc-4f73-a00d-0b324796e6fe admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:19:42.248758 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1015/2044] 10.222.0.23 () {60 vars in 1482 bytes} [Thu May 19 12:19:42 2022] PUT /identity/v3/projects/adc7c7b781734864a8f2977b38b85479/users/ac09aa384cd8426ba338b618e765c1ca/roles/9db5671c65d3414ea2682afa2ccf1403 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:19:42.258807 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d00f943-d68e-4ae5-b7e6-e555b6cbcebf None None] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:19:42.259101 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d00f943-d68e-4ae5-b7e6-e555b6cbcebf None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:19:42.259390 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d00f943-d68e-4ae5-b7e6-e555b6cbcebf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:19:42.306354 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.auth.core [None req-6d00f943-d68e-4ae5-b7e6-e555b6cbcebf None None] MFA Rules not processed for user `ac09aa384cd8426ba338b618e765c1ca`. Rule list: `[]` (Enabled: `True`). {{(pid=80540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:19:42.308060 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-6d00f943-d68e-4ae5-b7e6-e555b6cbcebf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:19:42.347917 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ad7ae3-5bfc-4f73-a00d-0b324796e6fe admin admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:19:42.349669 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-84ad7ae3-5bfc-4f73-a00d-0b324796e6fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:19:42.387705 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1030/2045] 10.222.0.23 () {58 vars in 957 bytes} [Thu May 19 12:19:42 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 12:19:42.403045 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d00f943-d68e-4ae5-b7e6-e555b6cbcebf None None] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:19:42.404761 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-6d00f943-d68e-4ae5-b7e6-e555b6cbcebf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:19:42.423498 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ad7ae3-5bfc-4f73-a00d-0b324796e6fe admin admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:19:42.429174 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5969dd-312f-4495-a27d-3837ff257b51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:19:42.429857 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5969dd-312f-4495-a27d-3837ff257b51 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:19:42.430304 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5969dd-312f-4495-a27d-3837ff257b51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:19:42.430592 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5969dd-312f-4495-a27d-3837ff257b51 admin admin] PATH_INFO: `/v3/users/5030c4dd874c432b8430986f9809b7b6` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:19:42.432026 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5969dd-312f-4495-a27d-3837ff257b51 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5030c4dd874c432b8430986f9809b7b6)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:19:42.434016 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5969dd-312f-4495-a27d-3837ff257b51 admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:19:42.435826 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:19:42.473350 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:19:42.480029 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d00f943-d68e-4ae5-b7e6-e555b6cbcebf None None] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:19:42.485051 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68375c7c-39a6-47b6-adee-d0c11c27aa5b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8800254c0e784e42a791fb9abc274d79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd70d25bf69314820b8aaecb65576978a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:19:42.485051 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68375c7c-39a6-47b6-adee-d0c11c27aa5b service cinder] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:19:42.485611 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68375c7c-39a6-47b6-adee-d0c11c27aa5b service cinder] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:19:42.485611 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68375c7c-39a6-47b6-adee-d0c11c27aa5b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:19:42.487454 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-68375c7c-39a6-47b6-adee-d0c11c27aa5b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:19:42.549587 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68375c7c-39a6-47b6-adee-d0c11c27aa5b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:19:42.551114 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68375c7c-39a6-47b6-adee-d0c11c27aa5b service cinder] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:19:42.561148 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1031/2046] 10.222.0.23 () {60 vars in 1365 bytes} [Thu May 19 12:19:42 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 12:19:42.561977 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68375c7c-39a6-47b6-adee-d0c11c27aa5b service cinder] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:19:42.563339 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-68375c7c-39a6-47b6-adee-d0c11c27aa5b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:19:42.586527 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68375c7c-39a6-47b6-adee-d0c11c27aa5b service cinder] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:19:42.592766 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3782609-c0ae-4440-baaa-63fd5349aacf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:19:42.592766 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3782609-c0ae-4440-baaa-63fd5349aacf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:19:42.592766 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3782609-c0ae-4440-baaa-63fd5349aacf admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:19:42.593071 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3782609-c0ae-4440-baaa-63fd5349aacf admin admin] PATH_INFO: `/v3/projects/ac1fa41b6de74fd7a4f1583a2bb1ae54` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:19:42.594238 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3782609-c0ae-4440-baaa-63fd5349aacf admin admin] RBAC: Authorizing `identity:delete_project(project_id=ac1fa41b6de74fd7a4f1583a2bb1ae54)` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:19:42.596374 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3782609-c0ae-4440-baaa-63fd5349aacf admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:19:42.667382 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-0f5969dd-312f-4495-a27d-3837ff257b51 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5030c4dd874c432b8430986f9809b7b6'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:19:42.682899 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-0f5969dd-312f-4495-a27d-3837ff257b51 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5030c4dd874c432b8430986f9809b7b6'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:19:42.683639 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-0f5969dd-312f-4495-a27d-3837ff257b51 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5030c4dd874c432b8430986f9809b7b6'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:19:42.733585 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-f3782609-c0ae-4440-baaa-63fd5349aacf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ac1fa41b6de74fd7a4f1583a2bb1ae54', 'request_id': 'req-f3782609-c0ae-4440-baaa-63fd5349aacf'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:19:42.739916 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-f3782609-c0ae-4440-baaa-63fd5349aacf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ac1fa41b6de74fd7a4f1583a2bb1ae54', 'request_id': 'req-f3782609-c0ae-4440-baaa-63fd5349aacf'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:19:42.746279 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-f3782609-c0ae-4440-baaa-63fd5349aacf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ac1fa41b6de74fd7a4f1583a2bb1ae54', 'request_id': 'req-f3782609-c0ae-4440-baaa-63fd5349aacf'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:19:42.771081 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-0f5969dd-312f-4495-a27d-3837ff257b51 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5030c4dd874c432b8430986f9809b7b6'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:19:42.781787 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-0f5969dd-312f-4495-a27d-3837ff257b51 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5030c4dd874c432b8430986f9809b7b6'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:19:42.788092 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1016/2047] 10.222.0.23 () {58 vars in 1223 bytes} [Thu May 19 12:19:42 2022] DELETE /identity/v3/users/5030c4dd874c432b8430986f9809b7b6 => generated 0 bytes in 441 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 12:19:42.792554 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5d5584-956a-45dc-a348-8b6d8973657c None None] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:19:42.792843 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5d5584-956a-45dc-a348-8b6d8973657c None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:19:42.793124 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5d5584-956a-45dc-a348-8b6d8973657c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:19:42.811945 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.auth.core [None req-cf5d5584-956a-45dc-a348-8b6d8973657c None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:19:42.813202 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-cf5d5584-956a-45dc-a348-8b6d8973657c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:19:42.862441 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-f3782609-c0ae-4440-baaa-63fd5349aacf admin admin] The token cache is being invalidate because project ac1fa41b6de74fd7a4f1583a2bb1ae54 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80540) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 12:19:42.863619 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-f3782609-c0ae-4440-baaa-63fd5349aacf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:19:42.864974 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-f3782609-c0ae-4440-baaa-63fd5349aacf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:19:42.872097 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1032/2048] 10.222.0.23 () {58 vars in 1232 bytes} [Thu May 19 12:19:42 2022] DELETE /identity/v3/projects/ac1fa41b6de74fd7a4f1583a2bb1ae54 => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:19:42.874431 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d26dc12-a6f1-4c4b-8318-b2bfa13ff612 None None] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:19:42.875293 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d26dc12-a6f1-4c4b-8318-b2bfa13ff612 None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:19:42.876160 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d26dc12-a6f1-4c4b-8318-b2bfa13ff612 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:19:42.899722 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1017/2049] 10.222.0.23 () {58 vars in 957 bytes} [Thu May 19 12:19:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 12:19:42.900830 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5d5584-956a-45dc-a348-8b6d8973657c None None] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:19:42.902249 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-cf5d5584-956a-45dc-a348-8b6d8973657c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:19:42.907317 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.auth.core [None req-2d26dc12-a6f1-4c4b-8318-b2bfa13ff612 None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:19:42.909529 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-2d26dc12-a6f1-4c4b-8318-b2bfa13ff612 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:19:42.922411 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1033/2050] 10.222.0.23 () {58 vars in 957 bytes} [Thu May 19 12:19:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:19:42.931729 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86917c2a-be58-4d97-89db-7f939551f6a6 None None] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:19:42.932172 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86917c2a-be58-4d97-89db-7f939551f6a6 None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:19:42.932470 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86917c2a-be58-4d97-89db-7f939551f6a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:19:42.967858 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.auth.core [None req-86917c2a-be58-4d97-89db-7f939551f6a6 None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:19:42.967858 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-86917c2a-be58-4d97-89db-7f939551f6a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:19:42.970614 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1034/2051] 10.222.0.23 () {58 vars in 957 bytes} [Thu May 19 12:19:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:19:42.975328 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86917c2a-be58-4d97-89db-7f939551f6a6 None None] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:19:42.986991 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5d5584-956a-45dc-a348-8b6d8973657c None None] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:19:42.987248 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86917c2a-be58-4d97-89db-7f939551f6a6 None None] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:19:42.991180 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01b301d-57b5-4464-964c-4f0ff044510d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a00b3fa7497456895e5e0bf105c4fc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd70d25bf69314820b8aaecb65576978a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:19:42.992162 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01b301d-57b5-4464-964c-4f0ff044510d service neutron] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:19:42.992678 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01b301d-57b5-4464-964c-4f0ff044510d service neutron] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:19:42.992968 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01b301d-57b5-4464-964c-4f0ff044510d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:19:42.997582 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d377e1f1-b4a2-43c7-a4fe-f4e091b4dcd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:19:42.998478 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d377e1f1-b4a2-43c7-a4fe-f4e091b4dcd3 admin admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:19:42.998769 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d377e1f1-b4a2-43c7-a4fe-f4e091b4dcd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:19:42.999046 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d377e1f1-b4a2-43c7-a4fe-f4e091b4dcd3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:19:43.000125 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d377e1f1-b4a2-43c7-a4fe-f4e091b4dcd3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:19:43.001168 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d377e1f1-b4a2-43c7-a4fe-f4e091b4dcd3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:19:43.003549 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01b301d-57b5-4464-964c-4f0ff044510d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:19:43.005141 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01b301d-57b5-4464-964c-4f0ff044510d service neutron] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:19:43.012921 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d377e1f1-b4a2-43c7-a4fe-f4e091b4dcd3 admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:19:43.016994 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1018/2052] 10.222.0.23 () {60 vars in 1365 bytes} [Thu May 19 12:19:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:19:43.028951 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1035/2053] 10.222.0.23 () {58 vars in 1152 bytes} [Thu May 19 12:19:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 12:19:43.033258 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01b301d-57b5-4464-964c-4f0ff044510d service neutron] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:19:43.044615 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01b301d-57b5-4464-964c-4f0ff044510d service neutron] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:19:43.049876 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819c90ba-9a7b-49a9-884d-b2cda70867ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:19:43.050573 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819c90ba-9a7b-49a9-884d-b2cda70867ab admin admin] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:19:43.050855 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819c90ba-9a7b-49a9-884d-b2cda70867ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:19:43.051135 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819c90ba-9a7b-49a9-884d-b2cda70867ab admin admin] PATH_INFO: `/v3/projects` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:19:43.051853 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819c90ba-9a7b-49a9-884d-b2cda70867ab admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:19:43.053767 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819c90ba-9a7b-49a9-884d-b2cda70867ab admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:19:43.197544 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:19:43.204666 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1019/2054] 10.222.0.23 () {60 vars in 1152 bytes} [Thu May 19 12:19:43 2022] POST /identity/v3/projects => generated 401 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 12:19:43.208762 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d377e1f1-b4a2-43c7-a4fe-f4e091b4dcd3 admin admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:19:43.221557 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d377e1f1-b4a2-43c7-a4fe-f4e091b4dcd3 admin admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:19:43.386679 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:19:43.392887 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dead9a76-0f63-4fe3-b450-38ae7aadca3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:19:43.393730 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dead9a76-0f63-4fe3-b450-38ae7aadca3d admin admin] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:19:43.394088 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dead9a76-0f63-4fe3-b450-38ae7aadca3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:19:43.394554 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dead9a76-0f63-4fe3-b450-38ae7aadca3d admin admin] PATH_INFO: `/v3/users` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:19:43.395335 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dead9a76-0f63-4fe3-b450-38ae7aadca3d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:19:43.397427 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dead9a76-0f63-4fe3-b450-38ae7aadca3d admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:19:43.493433 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1036/2055] 10.222.0.23 () {60 vars in 1143 bytes} [Thu May 19 12:19:43 2022] POST /identity/v3/users => generated 301 bytes in 285 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 12:19:43.497933 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819c90ba-9a7b-49a9-884d-b2cda70867ab admin admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:19:43.511363 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819c90ba-9a7b-49a9-884d-b2cda70867ab admin admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:19:43.518271 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0d8d56-4cca-4abb-aa2b-7e7430561c4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:19:43.519162 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0d8d56-4cca-4abb-aa2b-7e7430561c4d admin admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:19:43.520276 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0d8d56-4cca-4abb-aa2b-7e7430561c4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:19:43.520609 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0d8d56-4cca-4abb-aa2b-7e7430561c4d admin admin] PATH_INFO: `/v3/roles` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:19:43.521511 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0d8d56-4cca-4abb-aa2b-7e7430561c4d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:19:43.524133 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0d8d56-4cca-4abb-aa2b-7e7430561c4d admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:19:43.542567 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1020/2056] 10.222.0.23 () {58 vars in 1121 bytes} [Thu May 19 12:19:43 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 12:19:43.543735 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dead9a76-0f63-4fe3-b450-38ae7aadca3d admin admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:19:43.561368 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dead9a76-0f63-4fe3-b450-38ae7aadca3d admin admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:19:43.566828 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5fb836-4e13-4f94-95dd-955bdb1bc526 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:19:43.567763 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5fb836-4e13-4f94-95dd-955bdb1bc526 admin admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:19:43.568245 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5fb836-4e13-4f94-95dd-955bdb1bc526 admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:19:43.568535 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5fb836-4e13-4f94-95dd-955bdb1bc526 admin admin] PATH_INFO: `/v3/roles` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:19:43.569278 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5fb836-4e13-4f94-95dd-955bdb1bc526 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:19:43.571482 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5fb836-4e13-4f94-95dd-955bdb1bc526 admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:19:43.582926 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1037/2057] 10.222.0.23 () {58 vars in 1121 bytes} [Thu May 19 12:19:43 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 12:19:43.587310 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0d8d56-4cca-4abb-aa2b-7e7430561c4d admin admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:19:43.598643 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0d8d56-4cca-4abb-aa2b-7e7430561c4d admin admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:19:43.606309 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5bec88-521c-414c-af76-e5ab83f9fb78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:19:43.606309 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5bec88-521c-414c-af76-e5ab83f9fb78 admin admin] REQUEST_METHOD: `PUT` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:19:43.606309 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5bec88-521c-414c-af76-e5ab83f9fb78 admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:19:43.606309 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5bec88-521c-414c-af76-e5ab83f9fb78 admin admin] PATH_INFO: `/v3/projects/f7322035a6274e80bbe5ed3859e83f41/users/b089fd8ed3a347c480b1c8ece3d65273/roles/259eb400cdb4439db72d14456ae9cd08` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:19:43.622475 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5bec88-521c-414c-af76-e5ab83f9fb78 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f7322035a6274e80bbe5ed3859e83f41, user_id=b089fd8ed3a347c480b1c8ece3d65273, role_id=259eb400cdb4439db72d14456ae9cd08)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:19:43.625298 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5bec88-521c-414c-af76-e5ab83f9fb78 admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:19:43.667508 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5fb836-4e13-4f94-95dd-955bdb1bc526 admin admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:19:43.670123 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-ba5fb836-4e13-4f94-95dd-955bdb1bc526 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:19:43.688474 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5fb836-4e13-4f94-95dd-955bdb1bc526 admin admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:19:43.718483 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1021/2058] 10.222.0.23 () {60 vars in 1482 bytes} [Thu May 19 12:19:43 2022] PUT /identity/v3/projects/f7322035a6274e80bbe5ed3859e83f41/users/b089fd8ed3a347c480b1c8ece3d65273/roles/259eb400cdb4439db72d14456ae9cd08 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 12:19:43.731448 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c43fb7-58df-4456-9723-c91ce7de00e3 None None] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:19:43.732003 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c43fb7-58df-4456-9723-c91ce7de00e3 None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:19:43.732497 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c43fb7-58df-4456-9723-c91ce7de00e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:19:43.790752 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.auth.core [None req-97c43fb7-58df-4456-9723-c91ce7de00e3 None None] MFA Rules not processed for user `b089fd8ed3a347c480b1c8ece3d65273`. Rule list: `[]` (Enabled: `True`). {{(pid=80539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:19:43.792643 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-97c43fb7-58df-4456-9723-c91ce7de00e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:19:43.839776 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be49b239-6b3a-4b65-a8b5-5de8ca4e2ac4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:19:43.841346 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be49b239-6b3a-4b65-a8b5-5de8ca4e2ac4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:19:43.841856 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be49b239-6b3a-4b65-a8b5-5de8ca4e2ac4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:19:43.842369 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be49b239-6b3a-4b65-a8b5-5de8ca4e2ac4 admin admin] PATH_INFO: `/v3/projects/d12b0dc76a4f47758123998bee63907d` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:19:43.844829 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be49b239-6b3a-4b65-a8b5-5de8ca4e2ac4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d12b0dc76a4f47758123998bee63907d)` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:19:43.848730 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be49b239-6b3a-4b65-a8b5-5de8ca4e2ac4 admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:19:43.878627 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1022/2059] 10.222.0.23 () {58 vars in 957 bytes} [Thu May 19 12:19:43 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:19:43.883971 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c43fb7-58df-4456-9723-c91ce7de00e3 None None] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:19:43.895667 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c43fb7-58df-4456-9723-c91ce7de00e3 None None] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:19:43.901909 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3acdccca-0a8d-49ee-95b1-58e924bf46b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:19:43.902609 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acdccca-0a8d-49ee-95b1-58e924bf46b2 admin admin] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:19:43.902880 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acdccca-0a8d-49ee-95b1-58e924bf46b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:19:43.903157 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acdccca-0a8d-49ee-95b1-58e924bf46b2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:19:43.903978 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3acdccca-0a8d-49ee-95b1-58e924bf46b2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:19:43.905945 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3acdccca-0a8d-49ee-95b1-58e924bf46b2 admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:19:44.043418 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1023/2060] 10.222.0.23 () {60 vars in 1152 bytes} [Thu May 19 12:19:43 2022] POST /identity/v3/projects => generated 401 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 12:19:44.048277 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3acdccca-0a8d-49ee-95b1-58e924bf46b2 admin admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:19:44.059527 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-be49b239-6b3a-4b65-a8b5-5de8ca4e2ac4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd12b0dc76a4f47758123998bee63907d', 'request_id': 'req-be49b239-6b3a-4b65-a8b5-5de8ca4e2ac4'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:19:44.068846 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-be49b239-6b3a-4b65-a8b5-5de8ca4e2ac4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd12b0dc76a4f47758123998bee63907d', 'request_id': 'req-be49b239-6b3a-4b65-a8b5-5de8ca4e2ac4'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:19:44.078082 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-be49b239-6b3a-4b65-a8b5-5de8ca4e2ac4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd12b0dc76a4f47758123998bee63907d', 'request_id': 'req-be49b239-6b3a-4b65-a8b5-5de8ca4e2ac4'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:19:44.137958 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3acdccca-0a8d-49ee-95b1-58e924bf46b2 admin admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:19:44.143376 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c19a44f-5044-4988-99a0-e2b1397bd4d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:19:44.144086 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c19a44f-5044-4988-99a0-e2b1397bd4d1 admin admin] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:19:44.144398 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c19a44f-5044-4988-99a0-e2b1397bd4d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:19:44.144679 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c19a44f-5044-4988-99a0-e2b1397bd4d1 admin admin] PATH_INFO: `/v3/users` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:19:44.145384 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c19a44f-5044-4988-99a0-e2b1397bd4d1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:19:44.147337 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c19a44f-5044-4988-99a0-e2b1397bd4d1 admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:19:44.180440 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-be49b239-6b3a-4b65-a8b5-5de8ca4e2ac4 admin admin] The token cache is being invalidate because project d12b0dc76a4f47758123998bee63907d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80540) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 12:19:44.180727 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-be49b239-6b3a-4b65-a8b5-5de8ca4e2ac4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:19:44.181748 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-be49b239-6b3a-4b65-a8b5-5de8ca4e2ac4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:19:44.183513 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1038/2061] 10.222.0.23 () {58 vars in 1232 bytes} [Thu May 19 12:19:43 2022] DELETE /identity/v3/projects/d12b0dc76a4f47758123998bee63907d => generated 0 bytes in 517 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 12:19:44.187457 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be49b239-6b3a-4b65-a8b5-5de8ca4e2ac4 admin admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:19:44.188972 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-be49b239-6b3a-4b65-a8b5-5de8ca4e2ac4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:19:44.256983 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be49b239-6b3a-4b65-a8b5-5de8ca4e2ac4 admin admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:19:44.262312 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e436e745-26e9-4b61-b628-bee1a6321c4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:19:44.262494 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1024/2062] 10.222.0.23 () {60 vars in 1143 bytes} [Thu May 19 12:19:44 2022] POST /identity/v3/users => generated 307 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 12:19:44.263508 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e436e745-26e9-4b61-b628-bee1a6321c4b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:19:44.263810 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e436e745-26e9-4b61-b628-bee1a6321c4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:19:44.264113 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e436e745-26e9-4b61-b628-bee1a6321c4b admin admin] PATH_INFO: `/v3/users/ef80da8aa6884ebfa994e921754a4338` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:19:44.265337 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e436e745-26e9-4b61-b628-bee1a6321c4b admin admin] RBAC: Authorizing `identity:delete_user(user_id=ef80da8aa6884ebfa994e921754a4338)` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:19:44.267286 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e436e745-26e9-4b61-b628-bee1a6321c4b admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:19:44.268788 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c19a44f-5044-4988-99a0-e2b1397bd4d1 admin admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:19:44.271557 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-0c19a44f-5044-4988-99a0-e2b1397bd4d1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:19:44.284876 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c19a44f-5044-4988-99a0-e2b1397bd4d1 admin admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:19:44.290476 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3cdf981-89be-4cf1-b948-9a5f18eee007 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:19:44.291159 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cdf981-89be-4cf1-b948-9a5f18eee007 admin admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:19:44.291469 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cdf981-89be-4cf1-b948-9a5f18eee007 admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:19:44.291723 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cdf981-89be-4cf1-b948-9a5f18eee007 admin admin] PATH_INFO: `/v3/roles` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:19:44.292508 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3cdf981-89be-4cf1-b948-9a5f18eee007 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:19:44.294160 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3cdf981-89be-4cf1-b948-9a5f18eee007 admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:19:44.306221 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1025/2063] 10.222.0.23 () {58 vars in 1121 bytes} [Thu May 19 12:19:44 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 19 12:19:44.310572 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3cdf981-89be-4cf1-b948-9a5f18eee007 admin admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:19:44.320973 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3cdf981-89be-4cf1-b948-9a5f18eee007 admin admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:19:44.325815 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d736681e-0d7b-4af7-97f4-a674e5d6f531 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:19:44.326714 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d736681e-0d7b-4af7-97f4-a674e5d6f531 admin admin] REQUEST_METHOD: `PUT` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:19:44.327028 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d736681e-0d7b-4af7-97f4-a674e5d6f531 admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:19:44.327302 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d736681e-0d7b-4af7-97f4-a674e5d6f531 admin admin] PATH_INFO: `/v3/projects/3749085b2e8d4adfa3067ccdfbffd023/users/78a1b6914c7d43b2b35ab6547c021116/roles/9db5671c65d3414ea2682afa2ccf1403` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:19:44.336653 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:19:44.343225 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d736681e-0d7b-4af7-97f4-a674e5d6f531 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3749085b2e8d4adfa3067ccdfbffd023, user_id=78a1b6914c7d43b2b35ab6547c021116, role_id=9db5671c65d3414ea2682afa2ccf1403)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:19:44.345916 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d736681e-0d7b-4af7-97f4-a674e5d6f531 admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:19:44.475670 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1026/2064] 10.222.0.23 () {60 vars in 1482 bytes} [Thu May 19 12:19:44 2022] PUT /identity/v3/projects/3749085b2e8d4adfa3067ccdfbffd023/users/78a1b6914c7d43b2b35ab6547c021116/roles/9db5671c65d3414ea2682afa2ccf1403 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 12:19:44.480181 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d736681e-0d7b-4af7-97f4-a674e5d6f531 admin admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:19:44.550070 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d736681e-0d7b-4af7-97f4-a674e5d6f531 admin admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:19:44.554791 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77f8e63-e3ed-4e49-b8db-d229069b00db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:19:44.555461 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77f8e63-e3ed-4e49-b8db-d229069b00db admin admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:19:44.555745 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77f8e63-e3ed-4e49-b8db-d229069b00db admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:19:44.556056 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77f8e63-e3ed-4e49-b8db-d229069b00db admin admin] PATH_INFO: `/v3/roles` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:19:44.556826 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77f8e63-e3ed-4e49-b8db-d229069b00db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:19:44.558533 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77f8e63-e3ed-4e49-b8db-d229069b00db admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:19:44.569678 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1027/2065] 10.222.0.23 () {58 vars in 1121 bytes} [Thu May 19 12:19:44 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 19 12:19:44.573800 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77f8e63-e3ed-4e49-b8db-d229069b00db admin admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:19:44.585336 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77f8e63-e3ed-4e49-b8db-d229069b00db admin admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:19:44.591333 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f050a29c-4d4f-4e61-b64f-518adfd05e97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:19:44.592629 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f050a29c-4d4f-4e61-b64f-518adfd05e97 admin admin] REQUEST_METHOD: `PUT` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:19:44.592893 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f050a29c-4d4f-4e61-b64f-518adfd05e97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:19:44.593143 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f050a29c-4d4f-4e61-b64f-518adfd05e97 admin admin] PATH_INFO: `/v3/projects/3749085b2e8d4adfa3067ccdfbffd023/users/78a1b6914c7d43b2b35ab6547c021116/roles/9db5671c65d3414ea2682afa2ccf1403` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:19:44.595268 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f050a29c-4d4f-4e61-b64f-518adfd05e97 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3749085b2e8d4adfa3067ccdfbffd023, user_id=78a1b6914c7d43b2b35ab6547c021116, role_id=9db5671c65d3414ea2682afa2ccf1403)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:19:44.598086 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f050a29c-4d4f-4e61-b64f-518adfd05e97 admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:19:44.611989 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1028/2066] 10.222.0.23 () {60 vars in 1482 bytes} [Thu May 19 12:19:44 2022] PUT /identity/v3/projects/3749085b2e8d4adfa3067ccdfbffd023/users/78a1b6914c7d43b2b35ab6547c021116/roles/9db5671c65d3414ea2682afa2ccf1403 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 12:19:44.622640 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-e436e745-26e9-4b61-b628-bee1a6321c4b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ef80da8aa6884ebfa994e921754a4338'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:19:44.623165 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8200526-20b2-4f22-96b2-3fc57bf39825 None None] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:19:44.623165 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8200526-20b2-4f22-96b2-3fc57bf39825 None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:19:44.623304 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8200526-20b2-4f22-96b2-3fc57bf39825 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:19:44.629234 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:19:44.644275 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-e436e745-26e9-4b61-b628-bee1a6321c4b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ef80da8aa6884ebfa994e921754a4338'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:19:44.646361 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-e436e745-26e9-4b61-b628-bee1a6321c4b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ef80da8aa6884ebfa994e921754a4338'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:19:44.659094 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.auth.core [None req-a8200526-20b2-4f22-96b2-3fc57bf39825 None None] MFA Rules not processed for user `78a1b6914c7d43b2b35ab6547c021116`. Rule list: `[]` (Enabled: `True`). {{(pid=80539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:19:44.664836 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-a8200526-20b2-4f22-96b2-3fc57bf39825 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:19:44.733210 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1029/2067] 10.222.0.23 () {58 vars in 957 bytes} [Thu May 19 12:19:44 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 12:19:44.744515 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8200526-20b2-4f22-96b2-3fc57bf39825 None None] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:19:44.745901 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-a8200526-20b2-4f22-96b2-3fc57bf39825 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:19:44.759978 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-e436e745-26e9-4b61-b628-bee1a6321c4b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ef80da8aa6884ebfa994e921754a4338'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:19:44.769535 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-e436e745-26e9-4b61-b628-bee1a6321c4b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ef80da8aa6884ebfa994e921754a4338'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:19:44.777689 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1039/2068] 10.222.0.23 () {58 vars in 1223 bytes} [Thu May 19 12:19:44 2022] DELETE /identity/v3/users/ef80da8aa6884ebfa994e921754a4338 => generated 0 bytes in 591 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 12:19:44.815864 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8200526-20b2-4f22-96b2-3fc57bf39825 None None] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:19:44.880643 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e377dc4a-053e-428a-b7e6-9c1fbb18d37d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8800254c0e784e42a791fb9abc274d79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd70d25bf69314820b8aaecb65576978a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:19:44.881450 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e377dc4a-053e-428a-b7e6-9c1fbb18d37d service cinder] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:19:44.881842 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e377dc4a-053e-428a-b7e6-9c1fbb18d37d service cinder] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:19:44.882230 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e377dc4a-053e-428a-b7e6-9c1fbb18d37d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:19:44.885446 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-e377dc4a-053e-428a-b7e6-9c1fbb18d37d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:19:44.995598 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e377dc4a-053e-428a-b7e6-9c1fbb18d37d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:19:44.997101 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e377dc4a-053e-428a-b7e6-9c1fbb18d37d service cinder] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:19:45.010643 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1030/2069] 10.222.0.23 () {60 vars in 1365 bytes} [Thu May 19 12:19:44 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 267 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 12:19:45.246106 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:19:45.382174 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e436e745-26e9-4b61-b628-bee1a6321c4b admin admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:19:45.384411 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-e436e745-26e9-4b61-b628-bee1a6321c4b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:19:45.400470 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e436e745-26e9-4b61-b628-bee1a6321c4b admin admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:19:45.405345 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a862ad2-2533-4f00-9217-4695ce89399a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8800254c0e784e42a791fb9abc274d79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd70d25bf69314820b8aaecb65576978a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:19:45.406359 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a862ad2-2533-4f00-9217-4695ce89399a service cinder] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:19:45.406799 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a862ad2-2533-4f00-9217-4695ce89399a service cinder] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:19:45.407244 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a862ad2-2533-4f00-9217-4695ce89399a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:19:45.407428 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e377dc4a-053e-428a-b7e6-9c1fbb18d37d service cinder] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:19:45.409385 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-1a862ad2-2533-4f00-9217-4695ce89399a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:19:45.414380 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-e377dc4a-053e-428a-b7e6-9c1fbb18d37d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:19:45.507150 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e377dc4a-053e-428a-b7e6-9c1fbb18d37d service cinder] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:19:45.512881 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4b7760-9951-4d74-8278-18e97bced5ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:19:45.518352 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4b7760-9951-4d74-8278-18e97bced5ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:19:45.518352 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4b7760-9951-4d74-8278-18e97bced5ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:19:45.518352 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4b7760-9951-4d74-8278-18e97bced5ba admin admin] PATH_INFO: `/v3/projects/3cdd6be030344eb3a724ae23d1a72b70` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:19:45.518352 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4b7760-9951-4d74-8278-18e97bced5ba admin admin] RBAC: Authorizing `identity:delete_project(project_id=3cdd6be030344eb3a724ae23d1a72b70)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:19:45.518352 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4b7760-9951-4d74-8278-18e97bced5ba admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:19:45.520162 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a862ad2-2533-4f00-9217-4695ce89399a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:19:45.521640 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a862ad2-2533-4f00-9217-4695ce89399a service cinder] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:19:45.532979 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1040/2070] 10.222.0.23 () {60 vars in 1365 bytes} [Thu May 19 12:19:45 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:19:45.650345 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-0b4b7760-9951-4d74-8278-18e97bced5ba admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3cdd6be030344eb3a724ae23d1a72b70', 'request_id': 'req-0b4b7760-9951-4d74-8278-18e97bced5ba'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:19:45.667261 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-0b4b7760-9951-4d74-8278-18e97bced5ba admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3cdd6be030344eb3a724ae23d1a72b70', 'request_id': 'req-0b4b7760-9951-4d74-8278-18e97bced5ba'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:19:45.782486 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-0b4b7760-9951-4d74-8278-18e97bced5ba admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3cdd6be030344eb3a724ae23d1a72b70', 'request_id': 'req-0b4b7760-9951-4d74-8278-18e97bced5ba'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:19:45.794494 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-0b4b7760-9951-4d74-8278-18e97bced5ba admin admin] The token cache is being invalidate because project 3cdd6be030344eb3a724ae23d1a72b70 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80539) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 12:19:45.794789 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-0b4b7760-9951-4d74-8278-18e97bced5ba admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:19:45.795493 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-0b4b7760-9951-4d74-8278-18e97bced5ba admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:19:45.797076 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1031/2071] 10.222.0.23 () {58 vars in 1232 bytes} [Thu May 19 12:19:45 2022] DELETE /identity/v3/projects/3cdd6be030344eb3a724ae23d1a72b70 => generated 0 bytes in 390 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:19:45.802539 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b92de60-fb0d-4286-8f8d-a3261a5d8fb8 None None] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:19:45.803020 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b92de60-fb0d-4286-8f8d-a3261a5d8fb8 None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:19:45.803112 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b92de60-fb0d-4286-8f8d-a3261a5d8fb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:19:45.822916 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.auth.core [None req-3b92de60-fb0d-4286-8f8d-a3261a5d8fb8 None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:19:45.824201 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-3b92de60-fb0d-4286-8f8d-a3261a5d8fb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:19:45.885579 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1041/2072] 10.222.0.23 () {58 vars in 957 bytes} [Thu May 19 12:19:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:19:45.893470 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca02da29-9873-4d16-8f23-464f8c908f23 None None] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:19:45.893686 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca02da29-9873-4d16-8f23-464f8c908f23 None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:19:45.893963 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca02da29-9873-4d16-8f23-464f8c908f23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:19:45.916314 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.auth.core [None req-ca02da29-9873-4d16-8f23-464f8c908f23 None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:19:45.918631 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-ca02da29-9873-4d16-8f23-464f8c908f23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:19:45.926534 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1032/2073] 10.222.0.23 () {58 vars in 957 bytes} [Thu May 19 12:19:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:19:45.931081 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b92de60-fb0d-4286-8f8d-a3261a5d8fb8 None None] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:19:45.942437 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b92de60-fb0d-4286-8f8d-a3261a5d8fb8 None None] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:19:45.949315 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2e0fc8-aa69-48ac-8ce3-ed9b084502fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:19:45.950021 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2e0fc8-aa69-48ac-8ce3-ed9b084502fd admin admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:19:45.950311 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2e0fc8-aa69-48ac-8ce3-ed9b084502fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:19:45.950594 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2e0fc8-aa69-48ac-8ce3-ed9b084502fd admin admin] PATH_INFO: `/v3/domains` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:19:45.951352 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2e0fc8-aa69-48ac-8ce3-ed9b084502fd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:19:45.951667 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2e0fc8-aa69-48ac-8ce3-ed9b084502fd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:19:45.953456 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2e0fc8-aa69-48ac-8ce3-ed9b084502fd admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:19:45.967871 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1042/2074] 10.222.0.23 () {58 vars in 1152 bytes} [Thu May 19 12:19:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:19:45.972121 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca02da29-9873-4d16-8f23-464f8c908f23 None None] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:19:45.982222 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca02da29-9873-4d16-8f23-464f8c908f23 None None] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:19:45.987597 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85f0242-0599-4859-aa3d-0be0d5195da2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:19:45.988372 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85f0242-0599-4859-aa3d-0be0d5195da2 admin admin] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:19:45.989072 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85f0242-0599-4859-aa3d-0be0d5195da2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:19:45.989072 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85f0242-0599-4859-aa3d-0be0d5195da2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:19:45.989584 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85f0242-0599-4859-aa3d-0be0d5195da2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:19:45.991500 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85f0242-0599-4859-aa3d-0be0d5195da2 admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:19:46.090324 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2e0fc8-aa69-48ac-8ce3-ed9b084502fd admin admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:19:46.092074 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-5f2e0fc8-aa69-48ac-8ce3-ed9b084502fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:19:46.107201 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:19:46.111092 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1033/2075] 10.222.0.23 () {60 vars in 1152 bytes} [Thu May 19 12:19:45 2022] POST /identity/v3/projects => generated 383 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 12:19:46.115149 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85f0242-0599-4859-aa3d-0be0d5195da2 admin admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:19:46.159235 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2e0fc8-aa69-48ac-8ce3-ed9b084502fd admin admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:19:46.185318 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85f0242-0599-4859-aa3d-0be0d5195da2 admin admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:19:46.190867 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4535afee-3f6c-4597-8c17-c064ffd02e3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:19:46.191544 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4535afee-3f6c-4597-8c17-c064ffd02e3f admin admin] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:19:46.191850 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4535afee-3f6c-4597-8c17-c064ffd02e3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:19:46.192133 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4535afee-3f6c-4597-8c17-c064ffd02e3f admin admin] PATH_INFO: `/v3/users` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:19:46.196146 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4535afee-3f6c-4597-8c17-c064ffd02e3f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:19:46.198317 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4535afee-3f6c-4597-8c17-c064ffd02e3f admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:19:46.227853 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f004e2-7e8b-4e78-827a-daeb23a4e07c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8800254c0e784e42a791fb9abc274d79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd70d25bf69314820b8aaecb65576978a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:19:46.228584 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f004e2-7e8b-4e78-827a-daeb23a4e07c service cinder] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:19:46.228848 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f004e2-7e8b-4e78-827a-daeb23a4e07c service cinder] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:19:46.229393 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f004e2-7e8b-4e78-827a-daeb23a4e07c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:19:46.231292 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-d4f004e2-7e8b-4e78-827a-daeb23a4e07c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:19:46.251098 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:19:46.299134 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1034/2076] 10.222.0.23 () {60 vars in 1143 bytes} [Thu May 19 12:19:46 2022] POST /identity/v3/users => generated 292 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 12:19:46.302623 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4535afee-3f6c-4597-8c17-c064ffd02e3f admin admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:19:46.311429 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f004e2-7e8b-4e78-827a-daeb23a4e07c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:19:46.313728 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f004e2-7e8b-4e78-827a-daeb23a4e07c service cinder] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:19:46.324997 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4535afee-3f6c-4597-8c17-c064ffd02e3f admin admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:19:46.330847 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1354a06-6599-442a-8b5a-7296758014ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:19:46.332283 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1354a06-6599-442a-8b5a-7296758014ae admin admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:19:46.332666 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1354a06-6599-442a-8b5a-7296758014ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:19:46.332967 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1354a06-6599-442a-8b5a-7296758014ae admin admin] PATH_INFO: `/v3/roles` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:19:46.335830 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1354a06-6599-442a-8b5a-7296758014ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:19:46.335830 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1354a06-6599-442a-8b5a-7296758014ae admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:19:46.336183 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1043/2077] 10.222.0.23 () {60 vars in 1365 bytes} [Thu May 19 12:19:46 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 246 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 12:19:46.348065 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1035/2078] 10.222.0.23 () {58 vars in 1121 bytes} [Thu May 19 12:19:46 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 19 12:19:46.352303 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f004e2-7e8b-4e78-827a-daeb23a4e07c service cinder] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:19:46.365732 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f004e2-7e8b-4e78-827a-daeb23a4e07c service cinder] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:19:46.374613 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb1a32e-2214-4836-9408-638860b1349e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:19:46.375424 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb1a32e-2214-4836-9408-638860b1349e admin admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:19:46.375714 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb1a32e-2214-4836-9408-638860b1349e admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:19:46.376037 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb1a32e-2214-4836-9408-638860b1349e admin admin] PATH_INFO: `/v3/roles` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:19:46.376880 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb1a32e-2214-4836-9408-638860b1349e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:19:46.378911 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb1a32e-2214-4836-9408-638860b1349e admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:19:46.396293 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1044/2079] 10.222.0.23 () {58 vars in 1121 bytes} [Thu May 19 12:19:46 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 19 12:19:46.397459 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1354a06-6599-442a-8b5a-7296758014ae admin admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:19:46.409049 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1354a06-6599-442a-8b5a-7296758014ae admin admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:19:46.415654 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec546eb-b2b7-4327-83e8-785289575ace admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:19:46.416519 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec546eb-b2b7-4327-83e8-785289575ace admin admin] REQUEST_METHOD: `PUT` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:19:46.425626 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec546eb-b2b7-4327-83e8-785289575ace admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:19:46.425626 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec546eb-b2b7-4327-83e8-785289575ace admin admin] PATH_INFO: `/v3/projects/c634fdb709ba49cc98edc1d7648cad29/users/4dff4a0d51144a65ba39e0ca97296627/roles/259eb400cdb4439db72d14456ae9cd08` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:19:46.457027 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec546eb-b2b7-4327-83e8-785289575ace admin admin] RBAC: Authorizing `identity:create_grant(project_id=c634fdb709ba49cc98edc1d7648cad29, user_id=4dff4a0d51144a65ba39e0ca97296627, role_id=259eb400cdb4439db72d14456ae9cd08)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:19:46.460029 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec546eb-b2b7-4327-83e8-785289575ace admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:19:46.562349 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1036/2080] 10.222.0.23 () {60 vars in 1482 bytes} [Thu May 19 12:19:46 2022] PUT /identity/v3/projects/c634fdb709ba49cc98edc1d7648cad29/users/4dff4a0d51144a65ba39e0ca97296627/roles/259eb400cdb4439db72d14456ae9cd08 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:19:46.580095 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bec73ba-c795-43ab-a40f-a7e66b60bce5 None None] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:19:46.580705 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bec73ba-c795-43ab-a40f-a7e66b60bce5 None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:19:46.581143 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bec73ba-c795-43ab-a40f-a7e66b60bce5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:19:46.611864 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.auth.core [None req-0bec73ba-c795-43ab-a40f-a7e66b60bce5 None None] MFA Rules not processed for user `4dff4a0d51144a65ba39e0ca97296627`. Rule list: `[]` (Enabled: `True`). {{(pid=80540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:19:46.613225 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-0bec73ba-c795-43ab-a40f-a7e66b60bce5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:19:46.678771 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1045/2081] 10.222.0.23 () {58 vars in 957 bytes} [Thu May 19 12:19:46 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:19:46.684389 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec546eb-b2b7-4327-83e8-785289575ace admin admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:19:46.776578 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec546eb-b2b7-4327-83e8-785289575ace admin admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:19:46.787038 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb16613c-34cc-4dc7-8bc4-9410bc54b060 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:19:46.788020 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb16613c-34cc-4dc7-8bc4-9410bc54b060 admin admin] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:19:46.789503 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb16613c-34cc-4dc7-8bc4-9410bc54b060 admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:19:46.790291 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb16613c-34cc-4dc7-8bc4-9410bc54b060 admin admin] PATH_INFO: `/v3/projects` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:19:46.791573 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb16613c-34cc-4dc7-8bc4-9410bc54b060 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:19:46.796135 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb16613c-34cc-4dc7-8bc4-9410bc54b060 admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:19:46.936696 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1037/2082] 10.222.0.23 () {60 vars in 1152 bytes} [Thu May 19 12:19:46 2022] POST /identity/v3/projects => generated 383 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 12:19:46.940852 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bec73ba-c795-43ab-a40f-a7e66b60bce5 None None] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:19:46.964457 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bec73ba-c795-43ab-a40f-a7e66b60bce5 None None] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:19:47.051497 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291453bb-99ee-4e14-889b-3dce45c1202e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:19:47.052212 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291453bb-99ee-4e14-889b-3dce45c1202e admin admin] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:19:47.052498 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291453bb-99ee-4e14-889b-3dce45c1202e admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:19:47.052778 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291453bb-99ee-4e14-889b-3dce45c1202e admin admin] PATH_INFO: `/v3/users` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:19:47.053478 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291453bb-99ee-4e14-889b-3dce45c1202e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:19:47.056508 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291453bb-99ee-4e14-889b-3dce45c1202e admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:19:47.148624 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1046/2083] 10.222.0.23 () {60 vars in 1143 bytes} [Thu May 19 12:19:46 2022] POST /identity/v3/users => generated 298 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 12:19:47.153240 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb16613c-34cc-4dc7-8bc4-9410bc54b060 admin admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:19:47.164176 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb16613c-34cc-4dc7-8bc4-9410bc54b060 admin admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:19:47.169178 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ad5f51-6e62-4be5-ba98-5a4ca864daf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:19:47.171654 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ad5f51-6e62-4be5-ba98-5a4ca864daf3 admin admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:19:47.172236 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ad5f51-6e62-4be5-ba98-5a4ca864daf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:19:47.172236 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ad5f51-6e62-4be5-ba98-5a4ca864daf3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:19:47.173017 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ad5f51-6e62-4be5-ba98-5a4ca864daf3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:19:47.174706 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ad5f51-6e62-4be5-ba98-5a4ca864daf3 admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:19:47.192826 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1038/2084] 10.222.0.23 () {58 vars in 1121 bytes} [Thu May 19 12:19:47 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 12:19:47.193482 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291453bb-99ee-4e14-889b-3dce45c1202e admin admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:19:47.208284 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291453bb-99ee-4e14-889b-3dce45c1202e admin admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:19:47.215232 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc526e07-f014-4820-a912-f31a6f6548d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:19:47.216231 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc526e07-f014-4820-a912-f31a6f6548d4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:19:47.216548 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc526e07-f014-4820-a912-f31a6f6548d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:19:47.216948 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc526e07-f014-4820-a912-f31a6f6548d4 admin admin] PATH_INFO: `/v3/projects/8b6467a9a31347a0b786120ff77cd8a0/users/a92fca67f0c14610a5d61bbfdc047775/roles/9db5671c65d3414ea2682afa2ccf1403` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:19:47.235940 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc526e07-f014-4820-a912-f31a6f6548d4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8b6467a9a31347a0b786120ff77cd8a0, user_id=a92fca67f0c14610a5d61bbfdc047775, role_id=9db5671c65d3414ea2682afa2ccf1403)` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:19:47.238768 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc526e07-f014-4820-a912-f31a6f6548d4 admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:19:47.355678 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ad5f51-6e62-4be5-ba98-5a4ca864daf3 admin admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:19:47.362392 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1047/2085] 10.222.0.23 () {60 vars in 1482 bytes} [Thu May 19 12:19:47 2022] PUT /identity/v3/projects/8b6467a9a31347a0b786120ff77cd8a0/users/a92fca67f0c14610a5d61bbfdc047775/roles/9db5671c65d3414ea2682afa2ccf1403 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:19:47.368522 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ad5f51-6e62-4be5-ba98-5a4ca864daf3 admin admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:19:47.435774 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f5bc13-4770-4bc3-8b1b-78c8fa8906f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:19:47.436550 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f5bc13-4770-4bc3-8b1b-78c8fa8906f6 admin admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:19:47.436864 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f5bc13-4770-4bc3-8b1b-78c8fa8906f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:19:47.437153 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f5bc13-4770-4bc3-8b1b-78c8fa8906f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:19:47.438363 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f5bc13-4770-4bc3-8b1b-78c8fa8906f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:19:47.439510 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f5bc13-4770-4bc3-8b1b-78c8fa8906f6 admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:19:47.464903 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc526e07-f014-4820-a912-f31a6f6548d4 admin admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:19:47.468087 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1039/2086] 10.222.0.23 () {58 vars in 1121 bytes} [Thu May 19 12:19:47 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 12:19:47.484131 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc526e07-f014-4820-a912-f31a6f6548d4 admin admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:19:47.494210 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a67d4c-14c4-474b-8a92-293549bb6779 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:19:47.495207 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a67d4c-14c4-474b-8a92-293549bb6779 admin admin] REQUEST_METHOD: `PUT` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:19:47.496686 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a67d4c-14c4-474b-8a92-293549bb6779 admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:19:47.497156 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a67d4c-14c4-474b-8a92-293549bb6779 admin admin] PATH_INFO: `/v3/projects/8b6467a9a31347a0b786120ff77cd8a0/users/a92fca67f0c14610a5d61bbfdc047775/roles/9db5671c65d3414ea2682afa2ccf1403` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:19:47.499922 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a67d4c-14c4-474b-8a92-293549bb6779 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8b6467a9a31347a0b786120ff77cd8a0, user_id=a92fca67f0c14610a5d61bbfdc047775, role_id=9db5671c65d3414ea2682afa2ccf1403)` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:19:47.502736 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a67d4c-14c4-474b-8a92-293549bb6779 admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:19:47.516913 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1048/2087] 10.222.0.23 () {60 vars in 1482 bytes} [Thu May 19 12:19:47 2022] PUT /identity/v3/projects/8b6467a9a31347a0b786120ff77cd8a0/users/a92fca67f0c14610a5d61bbfdc047775/roles/9db5671c65d3414ea2682afa2ccf1403 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:19:47.528085 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6c0b83-5f35-4141-9d26-99f31067ddc5 None None] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:19:47.528247 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6c0b83-5f35-4141-9d26-99f31067ddc5 None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:19:47.528532 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6c0b83-5f35-4141-9d26-99f31067ddc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:19:47.560879 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.auth.core [None req-bf6c0b83-5f35-4141-9d26-99f31067ddc5 None None] MFA Rules not processed for user `a92fca67f0c14610a5d61bbfdc047775`. Rule list: `[]` (Enabled: `True`). {{(pid=80539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:19:47.562192 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-bf6c0b83-5f35-4141-9d26-99f31067ddc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:19:47.641617 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1040/2088] 10.222.0.23 () {58 vars in 957 bytes} [Thu May 19 12:19:47 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:19:47.651731 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:19:47.653452 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a67d4c-14c4-474b-8a92-293549bb6779 admin admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:19:47.655375 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-05a67d4c-14c4-474b-8a92-293549bb6779 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:19:47.718308 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a67d4c-14c4-474b-8a92-293549bb6779 admin admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:19:47.722664 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e480c9c-3720-43cb-bcaa-7e855d1630d7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8800254c0e784e42a791fb9abc274d79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd70d25bf69314820b8aaecb65576978a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:19:47.727111 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e480c9c-3720-43cb-bcaa-7e855d1630d7 service cinder] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:19:47.727111 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e480c9c-3720-43cb-bcaa-7e855d1630d7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:19:47.727111 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e480c9c-3720-43cb-bcaa-7e855d1630d7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:19:47.735377 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e480c9c-3720-43cb-bcaa-7e855d1630d7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:19:47.737242 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e480c9c-3720-43cb-bcaa-7e855d1630d7 service cinder] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:19:47.748134 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1049/2089] 10.222.0.23 () {60 vars in 1365 bytes} [Thu May 19 12:19:47 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:19:47.864107 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6c0b83-5f35-4141-9d26-99f31067ddc5 None None] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:19:47.873152 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=93,keystone:DELETE=1,keystone:INSERT=6 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:19:47.877767 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6c0b83-5f35-4141-9d26-99f31067ddc5 None None] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:19:47.881980 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475f49c9-1564-420e-aa34-ff9bf83ec941 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8800254c0e784e42a791fb9abc274d79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd70d25bf69314820b8aaecb65576978a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:19:47.883383 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475f49c9-1564-420e-aa34-ff9bf83ec941 service cinder] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:19:47.883734 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475f49c9-1564-420e-aa34-ff9bf83ec941 service cinder] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:19:47.884100 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475f49c9-1564-420e-aa34-ff9bf83ec941 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:19:47.944679 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475f49c9-1564-420e-aa34-ff9bf83ec941 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:19:47.947263 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475f49c9-1564-420e-aa34-ff9bf83ec941 service cinder] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:19:47.958254 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1041/2090] 10.222.0.23 () {60 vars in 1365 bytes} [Thu May 19 12:19:47 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:19:48.561972 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e480c9c-3720-43cb-bcaa-7e855d1630d7 service cinder] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:19:48.573464 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e480c9c-3720-43cb-bcaa-7e855d1630d7 service cinder] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:19:48.578261 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff8ed10-dcd9-45e0-b226-3bbe595e0ca8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8800254c0e784e42a791fb9abc274d79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd70d25bf69314820b8aaecb65576978a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:19:48.586767 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff8ed10-dcd9-45e0-b226-3bbe595e0ca8 service cinder] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:19:48.586767 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff8ed10-dcd9-45e0-b226-3bbe595e0ca8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:19:48.586767 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff8ed10-dcd9-45e0-b226-3bbe595e0ca8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:19:48.586767 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-bff8ed10-dcd9-45e0-b226-3bbe595e0ca8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:19:48.651162 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:19:48.668175 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff8ed10-dcd9-45e0-b226-3bbe595e0ca8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:19:48.669757 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff8ed10-dcd9-45e0-b226-3bbe595e0ca8 service cinder] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:19:48.681714 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1050/2091] 10.222.0.23 () {60 vars in 1365 bytes} [Thu May 19 12:19:48 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:19:53.828797 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475f49c9-1564-420e-aa34-ff9bf83ec941 service cinder] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:19:53.848095 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff8ed10-dcd9-45e0-b226-3bbe595e0ca8 service cinder] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:19:53.850185 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-bff8ed10-dcd9-45e0-b226-3bbe595e0ca8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:19:53.850697 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:19:53.917336 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475f49c9-1564-420e-aa34-ff9bf83ec941 service cinder] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:19:53.922196 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5a538d-d782-40b3-a878-9492350b9742 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:19:53.923079 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5a538d-d782-40b3-a878-9492350b9742 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:19:53.923363 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5a538d-d782-40b3-a878-9492350b9742 admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:19:53.923729 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5a538d-d782-40b3-a878-9492350b9742 admin admin] PATH_INFO: `/v3/users/4dff4a0d51144a65ba39e0ca97296627` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:19:53.925039 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e5a538d-d782-40b3-a878-9492350b9742 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4dff4a0d51144a65ba39e0ca97296627)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:19:53.926971 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e5a538d-d782-40b3-a878-9492350b9742 admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:19:53.937544 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff8ed10-dcd9-45e0-b226-3bbe595e0ca8 service cinder] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:19:53.942550 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e000b4c7-e7e3-4196-a6cb-eb4b74e5dfc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:19:53.943935 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e000b4c7-e7e3-4196-a6cb-eb4b74e5dfc7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:19:53.944227 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e000b4c7-e7e3-4196-a6cb-eb4b74e5dfc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:19:53.948286 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e000b4c7-e7e3-4196-a6cb-eb4b74e5dfc7 admin admin] PATH_INFO: `/v3/users/b089fd8ed3a347c480b1c8ece3d65273` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:19:53.948286 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e000b4c7-e7e3-4196-a6cb-eb4b74e5dfc7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b089fd8ed3a347c480b1c8ece3d65273)` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:19:53.948485 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e000b4c7-e7e3-4196-a6cb-eb4b74e5dfc7 admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:19:54.198847 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-3e5a538d-d782-40b3-a878-9492350b9742 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4dff4a0d51144a65ba39e0ca97296627'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:19:54.216898 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-e000b4c7-e7e3-4196-a6cb-eb4b74e5dfc7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b089fd8ed3a347c480b1c8ece3d65273'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:19:54.218743 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-3e5a538d-d782-40b3-a878-9492350b9742 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4dff4a0d51144a65ba39e0ca97296627'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:19:54.219541 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-3e5a538d-d782-40b3-a878-9492350b9742 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4dff4a0d51144a65ba39e0ca97296627'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:19:54.236065 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-e000b4c7-e7e3-4196-a6cb-eb4b74e5dfc7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b089fd8ed3a347c480b1c8ece3d65273'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:19:54.236795 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-e000b4c7-e7e3-4196-a6cb-eb4b74e5dfc7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b089fd8ed3a347c480b1c8ece3d65273'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:19:54.400119 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-3e5a538d-d782-40b3-a878-9492350b9742 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4dff4a0d51144a65ba39e0ca97296627'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:19:54.410824 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-3e5a538d-d782-40b3-a878-9492350b9742 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4dff4a0d51144a65ba39e0ca97296627'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:19:54.418700 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1042/2092] 10.222.0.23 () {58 vars in 1223 bytes} [Thu May 19 12:19:53 2022] DELETE /identity/v3/users/4dff4a0d51144a65ba39e0ca97296627 => generated 0 bytes in 590 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:19:54.423129 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf303a8-9535-48bf-8f24-e6f338fc4df8 None None] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:19:54.423556 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf303a8-9535-48bf-8f24-e6f338fc4df8 None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:19:54.423935 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf303a8-9535-48bf-8f24-e6f338fc4df8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:19:54.446795 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.auth.core [None req-acf303a8-9535-48bf-8f24-e6f338fc4df8 None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:19:54.448278 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-acf303a8-9535-48bf-8f24-e6f338fc4df8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:19:54.509180 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-e000b4c7-e7e3-4196-a6cb-eb4b74e5dfc7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b089fd8ed3a347c480b1c8ece3d65273'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:19:54.517620 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-e000b4c7-e7e3-4196-a6cb-eb4b74e5dfc7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b089fd8ed3a347c480b1c8ece3d65273'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:19:54.525516 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1051/2093] 10.222.0.23 () {58 vars in 1223 bytes} [Thu May 19 12:19:53 2022] DELETE /identity/v3/users/b089fd8ed3a347c480b1c8ece3d65273 => generated 0 bytes in 679 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:19:54.528649 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1043/2094] 10.222.0.23 () {58 vars in 957 bytes} [Thu May 19 12:19:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 12:19:54.532047 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebe6aaa-8a8e-40bc-ab81-8cf49cb53f22 None None] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:19:54.532428 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebe6aaa-8a8e-40bc-ab81-8cf49cb53f22 None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:19:54.533338 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebe6aaa-8a8e-40bc-ab81-8cf49cb53f22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:19:54.537315 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf303a8-9535-48bf-8f24-e6f338fc4df8 None None] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:19:54.538874 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-acf303a8-9535-48bf-8f24-e6f338fc4df8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:19:54.559377 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.auth.core [None req-9ebe6aaa-8a8e-40bc-ab81-8cf49cb53f22 None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:19:54.560769 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-9ebe6aaa-8a8e-40bc-ab81-8cf49cb53f22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:19:54.594563 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:19:54.633682 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf303a8-9535-48bf-8f24-e6f338fc4df8 None None] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:19:54.641563 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1052/2095] 10.222.0.23 () {58 vars in 957 bytes} [Thu May 19 12:19:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:19:54.653239 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebe6aaa-8a8e-40bc-ab81-8cf49cb53f22 None None] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:19:54.654972 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-9ebe6aaa-8a8e-40bc-ab81-8cf49cb53f22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:19:54.675670 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:19:54.705957 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf943e31-1735-452b-862d-38566bde6b5d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a00b3fa7497456895e5e0bf105c4fc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd70d25bf69314820b8aaecb65576978a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:19:54.706875 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf943e31-1735-452b-862d-38566bde6b5d service neutron] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:19:54.707905 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf943e31-1735-452b-862d-38566bde6b5d service neutron] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:19:54.707905 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf943e31-1735-452b-862d-38566bde6b5d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:19:54.718951 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf943e31-1735-452b-862d-38566bde6b5d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:19:54.720651 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf943e31-1735-452b-862d-38566bde6b5d service neutron] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:19:54.733588 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1044/2096] 10.222.0.23 () {60 vars in 1365 bytes} [Thu May 19 12:19:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 12:19:54.737630 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebe6aaa-8a8e-40bc-ab81-8cf49cb53f22 None None] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:19:54.741374 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640e4fe8-fd41-4fc5-bd4c-8d03452479d9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a00b3fa7497456895e5e0bf105c4fc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd70d25bf69314820b8aaecb65576978a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:19:54.742781 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640e4fe8-fd41-4fc5-bd4c-8d03452479d9 service neutron] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:19:54.743151 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640e4fe8-fd41-4fc5-bd4c-8d03452479d9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:19:54.743338 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640e4fe8-fd41-4fc5-bd4c-8d03452479d9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:19:54.753648 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640e4fe8-fd41-4fc5-bd4c-8d03452479d9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:19:54.755123 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640e4fe8-fd41-4fc5-bd4c-8d03452479d9 service neutron] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:19:54.766424 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1053/2097] 10.222.0.23 () {60 vars in 1365 bytes} [Thu May 19 12:19:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 12:19:54.995999 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf943e31-1735-452b-862d-38566bde6b5d service neutron] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:19:54.998058 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-bf943e31-1735-452b-862d-38566bde6b5d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:19:55.018196 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf943e31-1735-452b-862d-38566bde6b5d service neutron] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:19:55.024150 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab69ede-0763-4af0-a4c4-7e9c683c36e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:19:55.025203 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab69ede-0763-4af0-a4c4-7e9c683c36e2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:19:55.025566 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab69ede-0763-4af0-a4c4-7e9c683c36e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:19:55.025903 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab69ede-0763-4af0-a4c4-7e9c683c36e2 admin admin] PATH_INFO: `/v3/users/05053998a7fc4fe296698ebc085a68bf` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:19:55.027331 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab69ede-0763-4af0-a4c4-7e9c683c36e2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=05053998a7fc4fe296698ebc085a68bf)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:19:55.029447 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab69ede-0763-4af0-a4c4-7e9c683c36e2 admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:19:55.141418 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640e4fe8-fd41-4fc5-bd4c-8d03452479d9 service neutron] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:19:55.142887 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-640e4fe8-fd41-4fc5-bd4c-8d03452479d9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:19:55.158581 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640e4fe8-fd41-4fc5-bd4c-8d03452479d9 service neutron] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:19:55.165753 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfe25de-c1cb-49b6-ab10-ba87dea47cbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:19:55.166686 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfe25de-c1cb-49b6-ab10-ba87dea47cbc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:19:55.166974 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfe25de-c1cb-49b6-ab10-ba87dea47cbc admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:19:55.167265 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfe25de-c1cb-49b6-ab10-ba87dea47cbc admin admin] PATH_INFO: `/v3/users/9e66a142d53340bb8bc4a0a9a9f48d59` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:19:55.169099 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dfe25de-c1cb-49b6-ab10-ba87dea47cbc admin admin] RBAC: Authorizing `identity:delete_user(user_id=9e66a142d53340bb8bc4a0a9a9f48d59)` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:19:55.171502 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dfe25de-c1cb-49b6-ab10-ba87dea47cbc admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:19:55.319248 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-8ab69ede-0763-4af0-a4c4-7e9c683c36e2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '05053998a7fc4fe296698ebc085a68bf'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:19:55.334813 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-8ab69ede-0763-4af0-a4c4-7e9c683c36e2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '05053998a7fc4fe296698ebc085a68bf'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:19:55.337011 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-8ab69ede-0763-4af0-a4c4-7e9c683c36e2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '05053998a7fc4fe296698ebc085a68bf'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:19:55.434225 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-8ab69ede-0763-4af0-a4c4-7e9c683c36e2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '05053998a7fc4fe296698ebc085a68bf'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:19:55.444661 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-8ab69ede-0763-4af0-a4c4-7e9c683c36e2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '05053998a7fc4fe296698ebc085a68bf'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:19:55.454278 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1045/2098] 10.222.0.23 () {58 vars in 1223 bytes} [Thu May 19 12:19:54 2022] DELETE /identity/v3/users/05053998a7fc4fe296698ebc085a68bf => generated 0 bytes in 459 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:19:55.455436 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab69ede-0763-4af0-a4c4-7e9c683c36e2 admin admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:19:55.457132 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-8ab69ede-0763-4af0-a4c4-7e9c683c36e2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:19:55.459617 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-6dfe25de-c1cb-49b6-ab10-ba87dea47cbc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9e66a142d53340bb8bc4a0a9a9f48d59'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:19:55.500692 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-6dfe25de-c1cb-49b6-ab10-ba87dea47cbc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9e66a142d53340bb8bc4a0a9a9f48d59'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:19:55.504330 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-6dfe25de-c1cb-49b6-ab10-ba87dea47cbc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9e66a142d53340bb8bc4a0a9a9f48d59'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:19:55.545863 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab69ede-0763-4af0-a4c4-7e9c683c36e2 admin admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:19:55.550931 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7768cd91-d6da-4b9c-8f37-762057fa220d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:19:55.551836 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7768cd91-d6da-4b9c-8f37-762057fa220d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:19:55.552118 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7768cd91-d6da-4b9c-8f37-762057fa220d admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:19:55.552395 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7768cd91-d6da-4b9c-8f37-762057fa220d admin admin] PATH_INFO: `/v3/projects/c634fdb709ba49cc98edc1d7648cad29` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:19:55.553583 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7768cd91-d6da-4b9c-8f37-762057fa220d admin admin] RBAC: Authorizing `identity:delete_project(project_id=c634fdb709ba49cc98edc1d7648cad29)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:19:55.555828 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7768cd91-d6da-4b9c-8f37-762057fa220d admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:19:55.558539 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:19:55.616259 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-6dfe25de-c1cb-49b6-ab10-ba87dea47cbc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9e66a142d53340bb8bc4a0a9a9f48d59'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:19:55.624338 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-6dfe25de-c1cb-49b6-ab10-ba87dea47cbc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9e66a142d53340bb8bc4a0a9a9f48d59'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:19:55.630820 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1054/2099] 10.222.0.23 () {58 vars in 1223 bytes} [Thu May 19 12:19:55 2022] DELETE /identity/v3/users/9e66a142d53340bb8bc4a0a9a9f48d59 => generated 0 bytes in 490 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 12:19:55.633366 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e510ae-494d-4791-8694-9e78bcb10e9e None None] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:19:55.633767 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e510ae-494d-4791-8694-9e78bcb10e9e None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:19:55.634148 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e510ae-494d-4791-8694-9e78bcb10e9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:19:55.661042 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.auth.core [None req-a2e510ae-494d-4791-8694-9e78bcb10e9e None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:19:55.662370 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-a2e510ae-494d-4791-8694-9e78bcb10e9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:19:55.721644 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-7768cd91-d6da-4b9c-8f37-762057fa220d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c634fdb709ba49cc98edc1d7648cad29', 'request_id': 'req-7768cd91-d6da-4b9c-8f37-762057fa220d'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:19:55.728447 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-7768cd91-d6da-4b9c-8f37-762057fa220d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c634fdb709ba49cc98edc1d7648cad29', 'request_id': 'req-7768cd91-d6da-4b9c-8f37-762057fa220d'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:19:55.740971 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1055/2100] 10.222.0.23 () {58 vars in 957 bytes} [Thu May 19 12:19:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:19:55.742043 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e510ae-494d-4791-8694-9e78bcb10e9e None None] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:19:55.744315 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-a2e510ae-494d-4791-8694-9e78bcb10e9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:19:55.760302 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e510ae-494d-4791-8694-9e78bcb10e9e None None] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:19:55.791498 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:19:55.831050 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d646ee-6eb0-4837-a287-424e700a1555 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:19:55.831956 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d646ee-6eb0-4837-a287-424e700a1555 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:19:55.832237 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d646ee-6eb0-4837-a287-424e700a1555 admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:19:55.832520 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d646ee-6eb0-4837-a287-424e700a1555 admin admin] PATH_INFO: `/v3/projects/f7322035a6274e80bbe5ed3859e83f41` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:19:55.833730 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d646ee-6eb0-4837-a287-424e700a1555 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f7322035a6274e80bbe5ed3859e83f41)` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:19:55.835714 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d646ee-6eb0-4837-a287-424e700a1555 admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:19:55.844656 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-7768cd91-d6da-4b9c-8f37-762057fa220d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c634fdb709ba49cc98edc1d7648cad29', 'request_id': 'req-7768cd91-d6da-4b9c-8f37-762057fa220d'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:19:55.857126 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-7768cd91-d6da-4b9c-8f37-762057fa220d admin admin] The token cache is being invalidate because project c634fdb709ba49cc98edc1d7648cad29 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80539) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 12:19:55.858467 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-7768cd91-d6da-4b9c-8f37-762057fa220d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:19:55.858467 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-7768cd91-d6da-4b9c-8f37-762057fa220d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:19:55.859698 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1046/2101] 10.222.0.23 () {58 vars in 1232 bytes} [Thu May 19 12:19:55 2022] DELETE /identity/v3/projects/c634fdb709ba49cc98edc1d7648cad29 => generated 0 bytes in 405 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:19:55.862843 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7768cd91-d6da-4b9c-8f37-762057fa220d admin admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:19:55.862843 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-7768cd91-d6da-4b9c-8f37-762057fa220d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:19:55.918040 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7768cd91-d6da-4b9c-8f37-762057fa220d admin admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:19:55.921686 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6be96f8-4af6-48e5-8e5a-f66d59a30723 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a00b3fa7497456895e5e0bf105c4fc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd70d25bf69314820b8aaecb65576978a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:19:55.922325 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6be96f8-4af6-48e5-8e5a-f66d59a30723 service neutron] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:19:55.922610 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6be96f8-4af6-48e5-8e5a-f66d59a30723 service neutron] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:19:55.922885 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6be96f8-4af6-48e5-8e5a-f66d59a30723 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:19:55.925208 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-f6be96f8-4af6-48e5-8e5a-f66d59a30723 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:19:55.939833 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6be96f8-4af6-48e5-8e5a-f66d59a30723 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:19:55.941345 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6be96f8-4af6-48e5-8e5a-f66d59a30723 service neutron] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:19:55.952933 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1047/2102] 10.222.0.23 () {60 vars in 1365 bytes} [Thu May 19 12:19:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:19:55.953795 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6be96f8-4af6-48e5-8e5a-f66d59a30723 service neutron] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:19:55.955673 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-f6be96f8-4af6-48e5-8e5a-f66d59a30723 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:19:55.968104 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6be96f8-4af6-48e5-8e5a-f66d59a30723 service neutron] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:19:55.972814 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011d98f6-74e4-42a7-9788-c33b9049b97c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:19:55.973701 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011d98f6-74e4-42a7-9788-c33b9049b97c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:19:55.978648 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011d98f6-74e4-42a7-9788-c33b9049b97c admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:19:55.978648 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011d98f6-74e4-42a7-9788-c33b9049b97c admin admin] PATH_INFO: `/v3/users/a92fca67f0c14610a5d61bbfdc047775` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:19:55.978648 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011d98f6-74e4-42a7-9788-c33b9049b97c admin admin] RBAC: Authorizing `identity:delete_user(user_id=a92fca67f0c14610a5d61bbfdc047775)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:19:55.979861 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011d98f6-74e4-42a7-9788-c33b9049b97c admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:19:56.031723 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-36d646ee-6eb0-4837-a287-424e700a1555 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f7322035a6274e80bbe5ed3859e83f41', 'request_id': 'req-36d646ee-6eb0-4837-a287-424e700a1555'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:19:56.041604 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-36d646ee-6eb0-4837-a287-424e700a1555 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f7322035a6274e80bbe5ed3859e83f41', 'request_id': 'req-36d646ee-6eb0-4837-a287-424e700a1555'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:19:56.050786 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-36d646ee-6eb0-4837-a287-424e700a1555 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f7322035a6274e80bbe5ed3859e83f41', 'request_id': 'req-36d646ee-6eb0-4837-a287-424e700a1555'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:19:56.133940 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-36d646ee-6eb0-4837-a287-424e700a1555 admin admin] The token cache is being invalidate because project f7322035a6274e80bbe5ed3859e83f41 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80540) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 12:19:56.134312 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-36d646ee-6eb0-4837-a287-424e700a1555 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:19:56.141295 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-36d646ee-6eb0-4837-a287-424e700a1555 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:19:56.141295 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1056/2103] 10.222.0.23 () {58 vars in 1232 bytes} [Thu May 19 12:19:55 2022] DELETE /identity/v3/projects/f7322035a6274e80bbe5ed3859e83f41 => generated 0 bytes in 395 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:19:56.141295 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d646ee-6eb0-4837-a287-424e700a1555 admin admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:19:56.141295 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-36d646ee-6eb0-4837-a287-424e700a1555 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:19:56.209112 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d646ee-6eb0-4837-a287-424e700a1555 admin admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:19:56.214628 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89afc34f-f683-4e8b-901b-6c59b8f41893 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:19:56.225487 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89afc34f-f683-4e8b-901b-6c59b8f41893 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:19:56.225487 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89afc34f-f683-4e8b-901b-6c59b8f41893 admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:19:56.225487 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89afc34f-f683-4e8b-901b-6c59b8f41893 admin admin] PATH_INFO: `/v3/projects/de8f3490bba84ff38b5940fe768ede25` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:19:56.225487 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89afc34f-f683-4e8b-901b-6c59b8f41893 admin admin] RBAC: Authorizing `identity:delete_project(project_id=de8f3490bba84ff38b5940fe768ede25)` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:19:56.225487 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89afc34f-f683-4e8b-901b-6c59b8f41893 admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:19:56.259810 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-011d98f6-74e4-42a7-9788-c33b9049b97c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a92fca67f0c14610a5d61bbfdc047775'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:19:56.272424 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-011d98f6-74e4-42a7-9788-c33b9049b97c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a92fca67f0c14610a5d61bbfdc047775'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:19:56.273439 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-011d98f6-74e4-42a7-9788-c33b9049b97c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a92fca67f0c14610a5d61bbfdc047775'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:19:56.400119 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-89afc34f-f683-4e8b-901b-6c59b8f41893 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'de8f3490bba84ff38b5940fe768ede25', 'request_id': 'req-89afc34f-f683-4e8b-901b-6c59b8f41893'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:19:56.406827 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-89afc34f-f683-4e8b-901b-6c59b8f41893 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'de8f3490bba84ff38b5940fe768ede25', 'request_id': 'req-89afc34f-f683-4e8b-901b-6c59b8f41893'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:19:56.412249 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-89afc34f-f683-4e8b-901b-6c59b8f41893 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'de8f3490bba84ff38b5940fe768ede25', 'request_id': 'req-89afc34f-f683-4e8b-901b-6c59b8f41893'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:19:56.422639 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-011d98f6-74e4-42a7-9788-c33b9049b97c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a92fca67f0c14610a5d61bbfdc047775'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:19:56.430745 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-011d98f6-74e4-42a7-9788-c33b9049b97c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a92fca67f0c14610a5d61bbfdc047775'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:19:56.436870 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1048/2104] 10.222.0.23 () {58 vars in 1223 bytes} [Thu May 19 12:19:55 2022] DELETE /identity/v3/users/a92fca67f0c14610a5d61bbfdc047775 => generated 0 bytes in 483 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:19:56.437642 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011d98f6-74e4-42a7-9788-c33b9049b97c admin admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:19:56.438984 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-011d98f6-74e4-42a7-9788-c33b9049b97c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:19:56.506489 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011d98f6-74e4-42a7-9788-c33b9049b97c admin admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:19:56.512315 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85fa11b-3cc8-4452-85fd-92c09dde8b4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:19:56.513135 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85fa11b-3cc8-4452-85fd-92c09dde8b4b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:19:56.513442 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85fa11b-3cc8-4452-85fd-92c09dde8b4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:19:56.513700 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85fa11b-3cc8-4452-85fd-92c09dde8b4b admin admin] PATH_INFO: `/v3/users/78a1b6914c7d43b2b35ab6547c021116` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:19:56.515062 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85fa11b-3cc8-4452-85fd-92c09dde8b4b admin admin] RBAC: Authorizing `identity:delete_user(user_id=78a1b6914c7d43b2b35ab6547c021116)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:19:56.517726 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85fa11b-3cc8-4452-85fd-92c09dde8b4b admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:19:56.525061 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-89afc34f-f683-4e8b-901b-6c59b8f41893 admin admin] The token cache is being invalidate because project de8f3490bba84ff38b5940fe768ede25 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80540) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 12:19:56.525432 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-89afc34f-f683-4e8b-901b-6c59b8f41893 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:19:56.526279 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-89afc34f-f683-4e8b-901b-6c59b8f41893 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:19:56.532639 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1057/2105] 10.222.0.23 () {58 vars in 1232 bytes} [Thu May 19 12:19:56 2022] DELETE /identity/v3/projects/de8f3490bba84ff38b5940fe768ede25 => generated 0 bytes in 391 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:19:56.532639 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89afc34f-f683-4e8b-901b-6c59b8f41893 admin admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:19:56.533670 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-89afc34f-f683-4e8b-901b-6c59b8f41893 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:19:56.611710 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89afc34f-f683-4e8b-901b-6c59b8f41893 admin admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:19:56.616455 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc547315-13e9-47ab-8859-798a69bf00a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:19:56.617829 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc547315-13e9-47ab-8859-798a69bf00a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:19:56.618115 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc547315-13e9-47ab-8859-798a69bf00a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:19:56.618395 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc547315-13e9-47ab-8859-798a69bf00a2 admin admin] PATH_INFO: `/v3/projects/01164d76ccfa447ba19e004349921bfb` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:19:56.619827 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc547315-13e9-47ab-8859-798a69bf00a2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=01164d76ccfa447ba19e004349921bfb)` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:19:56.621863 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc547315-13e9-47ab-8859-798a69bf00a2 admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:19:56.771008 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-bc547315-13e9-47ab-8859-798a69bf00a2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '01164d76ccfa447ba19e004349921bfb', 'request_id': 'req-bc547315-13e9-47ab-8859-798a69bf00a2'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:19:56.798165 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-f85fa11b-3cc8-4452-85fd-92c09dde8b4b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '78a1b6914c7d43b2b35ab6547c021116'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:19:56.798525 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-bc547315-13e9-47ab-8859-798a69bf00a2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '01164d76ccfa447ba19e004349921bfb', 'request_id': 'req-bc547315-13e9-47ab-8859-798a69bf00a2'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:19:56.810480 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-bc547315-13e9-47ab-8859-798a69bf00a2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '01164d76ccfa447ba19e004349921bfb', 'request_id': 'req-bc547315-13e9-47ab-8859-798a69bf00a2'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:19:56.824299 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:19:56.841985 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-f85fa11b-3cc8-4452-85fd-92c09dde8b4b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '78a1b6914c7d43b2b35ab6547c021116'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:19:56.843344 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-f85fa11b-3cc8-4452-85fd-92c09dde8b4b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '78a1b6914c7d43b2b35ab6547c021116'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:19:56.902628 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-bc547315-13e9-47ab-8859-798a69bf00a2 admin admin] The token cache is being invalidate because project 01164d76ccfa447ba19e004349921bfb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80540) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 12:19:56.903017 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-bc547315-13e9-47ab-8859-798a69bf00a2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:19:56.903962 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-bc547315-13e9-47ab-8859-798a69bf00a2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:19:56.908839 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1058/2106] 10.222.0.23 () {58 vars in 1232 bytes} [Thu May 19 12:19:56 2022] DELETE /identity/v3/projects/01164d76ccfa447ba19e004349921bfb => generated 0 bytes in 377 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:19:56.912307 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8f1acf-ea76-4b1a-8567-f09a2345c613 None None] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:19:56.912307 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8f1acf-ea76-4b1a-8567-f09a2345c613 None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:19:56.912307 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8f1acf-ea76-4b1a-8567-f09a2345c613 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:19:56.938866 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.auth.core [None req-3f8f1acf-ea76-4b1a-8567-f09a2345c613 None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:19:56.940134 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-3f8f1acf-ea76-4b1a-8567-f09a2345c613 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:19:56.984537 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-f85fa11b-3cc8-4452-85fd-92c09dde8b4b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '78a1b6914c7d43b2b35ab6547c021116'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:19:56.993674 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-f85fa11b-3cc8-4452-85fd-92c09dde8b4b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '78a1b6914c7d43b2b35ab6547c021116'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:19:57.003548 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1049/2107] 10.222.0.23 () {58 vars in 1223 bytes} [Thu May 19 12:19:56 2022] DELETE /identity/v3/users/78a1b6914c7d43b2b35ab6547c021116 => generated 0 bytes in 563 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:19:57.003548 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85fa11b-3cc8-4452-85fd-92c09dde8b4b admin admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:19:57.009951 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-f85fa11b-3cc8-4452-85fd-92c09dde8b4b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:19:57.042868 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1059/2108] 10.222.0.23 () {58 vars in 957 bytes} [Thu May 19 12:19:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:19:57.044129 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8f1acf-ea76-4b1a-8567-f09a2345c613 None None] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:19:57.045799 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-3f8f1acf-ea76-4b1a-8567-f09a2345c613 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:19:57.060729 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8f1acf-ea76-4b1a-8567-f09a2345c613 None None] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:19:57.108276 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85fa11b-3cc8-4452-85fd-92c09dde8b4b admin admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:19:57.113731 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421f230e-9bb7-410f-ae3d-a64368091d44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:19:57.114942 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421f230e-9bb7-410f-ae3d-a64368091d44 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:19:57.115386 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421f230e-9bb7-410f-ae3d-a64368091d44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:19:57.115842 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421f230e-9bb7-410f-ae3d-a64368091d44 admin admin] PATH_INFO: `/v3/users/cbf69910293445a6b091ce90b1df1709` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:19:57.117400 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421f230e-9bb7-410f-ae3d-a64368091d44 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cbf69910293445a6b091ce90b1df1709)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:19:57.119475 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421f230e-9bb7-410f-ae3d-a64368091d44 admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:19:57.135852 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9dcf01-e651-4342-ae13-ee5eb38529f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:19:57.136785 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9dcf01-e651-4342-ae13-ee5eb38529f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:19:57.137131 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9dcf01-e651-4342-ae13-ee5eb38529f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:19:57.146131 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9dcf01-e651-4342-ae13-ee5eb38529f3 admin admin] PATH_INFO: `/v3/projects/8b6467a9a31347a0b786120ff77cd8a0` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:19:57.146131 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa9dcf01-e651-4342-ae13-ee5eb38529f3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8b6467a9a31347a0b786120ff77cd8a0)` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:19:57.146131 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa9dcf01-e651-4342-ae13-ee5eb38529f3 admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:19:57.419143 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-421f230e-9bb7-410f-ae3d-a64368091d44 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cbf69910293445a6b091ce90b1df1709'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:19:57.420055 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-aa9dcf01-e651-4342-ae13-ee5eb38529f3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8b6467a9a31347a0b786120ff77cd8a0', 'request_id': 'req-aa9dcf01-e651-4342-ae13-ee5eb38529f3'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:19:57.426507 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-aa9dcf01-e651-4342-ae13-ee5eb38529f3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8b6467a9a31347a0b786120ff77cd8a0', 'request_id': 'req-aa9dcf01-e651-4342-ae13-ee5eb38529f3'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:19:57.432656 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-421f230e-9bb7-410f-ae3d-a64368091d44 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cbf69910293445a6b091ce90b1df1709'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:19:57.432925 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-aa9dcf01-e651-4342-ae13-ee5eb38529f3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8b6467a9a31347a0b786120ff77cd8a0', 'request_id': 'req-aa9dcf01-e651-4342-ae13-ee5eb38529f3'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:19:57.440861 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-421f230e-9bb7-410f-ae3d-a64368091d44 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cbf69910293445a6b091ce90b1df1709'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:19:57.551950 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-aa9dcf01-e651-4342-ae13-ee5eb38529f3 admin admin] The token cache is being invalidate because project 8b6467a9a31347a0b786120ff77cd8a0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80540) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 12:19:57.553262 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-aa9dcf01-e651-4342-ae13-ee5eb38529f3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:19:57.554756 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-aa9dcf01-e651-4342-ae13-ee5eb38529f3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:19:57.557002 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1060/2109] 10.222.0.23 () {58 vars in 1232 bytes} [Thu May 19 12:19:57 2022] DELETE /identity/v3/projects/8b6467a9a31347a0b786120ff77cd8a0 => generated 0 bytes in 514 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:19:57.558414 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffbe67e-e89d-4d86-b4b8-2deae428a6e2 None None] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:19:57.559029 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffbe67e-e89d-4d86-b4b8-2deae428a6e2 None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:19:57.559318 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffbe67e-e89d-4d86-b4b8-2deae428a6e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:19:57.588854 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.auth.core [None req-0ffbe67e-e89d-4d86-b4b8-2deae428a6e2 None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:19:57.588854 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-0ffbe67e-e89d-4d86-b4b8-2deae428a6e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:19:57.639575 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-421f230e-9bb7-410f-ae3d-a64368091d44 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cbf69910293445a6b091ce90b1df1709'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:19:57.646226 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1061/2110] 10.222.0.23 () {58 vars in 957 bytes} [Thu May 19 12:19:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:19:57.647093 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffbe67e-e89d-4d86-b4b8-2deae428a6e2 None None] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:19:57.647283 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-421f230e-9bb7-410f-ae3d-a64368091d44 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cbf69910293445a6b091ce90b1df1709'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:19:57.648459 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-0ffbe67e-e89d-4d86-b4b8-2deae428a6e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:19:57.656002 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1050/2111] 10.222.0.23 () {58 vars in 1223 bytes} [Thu May 19 12:19:57 2022] DELETE /identity/v3/users/cbf69910293445a6b091ce90b1df1709 => generated 0 bytes in 654 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:19:57.660346 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffbe67e-e89d-4d86-b4b8-2deae428a6e2 None None] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:19:57.660527 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b5dc39-9158-4fa2-9903-7d6a0e5a3b05 None None] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:19:57.660705 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b5dc39-9158-4fa2-9903-7d6a0e5a3b05 None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:19:57.661101 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b5dc39-9158-4fa2-9903-7d6a0e5a3b05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:19:57.676486 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:19:57.683613 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.auth.core [None req-c8b5dc39-9158-4fa2-9903-7d6a0e5a3b05 None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:19:57.685819 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-c8b5dc39-9158-4fa2-9903-7d6a0e5a3b05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:19:57.741084 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4adc1c34-8e00-48e5-96f7-dfdfc5873655 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:19:57.742709 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adc1c34-8e00-48e5-96f7-dfdfc5873655 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:19:57.743002 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adc1c34-8e00-48e5-96f7-dfdfc5873655 admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:19:57.743281 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adc1c34-8e00-48e5-96f7-dfdfc5873655 admin admin] PATH_INFO: `/v3/projects/3749085b2e8d4adfa3067ccdfbffd023` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:19:57.744901 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4adc1c34-8e00-48e5-96f7-dfdfc5873655 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3749085b2e8d4adfa3067ccdfbffd023)` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:19:57.746873 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4adc1c34-8e00-48e5-96f7-dfdfc5873655 admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:19:57.760343 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1051/2112] 10.222.0.23 () {58 vars in 957 bytes} [Thu May 19 12:19:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:19:57.761629 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b5dc39-9158-4fa2-9903-7d6a0e5a3b05 None None] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:19:57.768333 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-c8b5dc39-9158-4fa2-9903-7d6a0e5a3b05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:19:57.783407 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b5dc39-9158-4fa2-9903-7d6a0e5a3b05 None None] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:19:57.788364 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5fa94ae-1ab6-4cba-81d2-490a6228a068 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:19:57.789047 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fa94ae-1ab6-4cba-81d2-490a6228a068 admin admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:19:57.789314 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fa94ae-1ab6-4cba-81d2-490a6228a068 admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:19:57.789723 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fa94ae-1ab6-4cba-81d2-490a6228a068 admin admin] PATH_INFO: `/v3/domains` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:19:57.790476 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5fa94ae-1ab6-4cba-81d2-490a6228a068 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:19:57.790790 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5fa94ae-1ab6-4cba-81d2-490a6228a068 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:19:57.792461 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5fa94ae-1ab6-4cba-81d2-490a6228a068 admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:19:57.804993 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1052/2113] 10.222.0.23 () {58 vars in 1152 bytes} [Thu May 19 12:19:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:19:57.806250 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc8a3e3-9500-4e3f-8832-6b7c2830db9a None None] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:19:57.806551 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc8a3e3-9500-4e3f-8832-6b7c2830db9a None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:19:57.806834 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc8a3e3-9500-4e3f-8832-6b7c2830db9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:19:57.826121 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.auth.core [None req-acc8a3e3-9500-4e3f-8832-6b7c2830db9a None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:19:57.827556 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-acc8a3e3-9500-4e3f-8832-6b7c2830db9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:19:57.840210 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1053/2114] 10.222.0.23 () {58 vars in 957 bytes} [Thu May 19 12:19:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:19:57.840210 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc8a3e3-9500-4e3f-8832-6b7c2830db9a None None] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:19:57.855606 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc8a3e3-9500-4e3f-8832-6b7c2830db9a None None] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:19:57.861442 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4ab36a-c900-4e99-abcf-ea911cb62d1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:19:57.862705 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4ab36a-c900-4e99-abcf-ea911cb62d1e admin admin] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:19:57.863218 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4ab36a-c900-4e99-abcf-ea911cb62d1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:19:57.863511 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4ab36a-c900-4e99-abcf-ea911cb62d1e admin admin] PATH_INFO: `/v3/projects` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:19:57.864560 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4ab36a-c900-4e99-abcf-ea911cb62d1e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:19:57.866823 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4ab36a-c900-4e99-abcf-ea911cb62d1e admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:19:57.920403 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-4adc1c34-8e00-48e5-96f7-dfdfc5873655 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3749085b2e8d4adfa3067ccdfbffd023', 'request_id': 'req-4adc1c34-8e00-48e5-96f7-dfdfc5873655'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:19:57.928671 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-4adc1c34-8e00-48e5-96f7-dfdfc5873655 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3749085b2e8d4adfa3067ccdfbffd023', 'request_id': 'req-4adc1c34-8e00-48e5-96f7-dfdfc5873655'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:19:57.935141 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-4adc1c34-8e00-48e5-96f7-dfdfc5873655 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3749085b2e8d4adfa3067ccdfbffd023', 'request_id': 'req-4adc1c34-8e00-48e5-96f7-dfdfc5873655'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:19:58.014895 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1054/2115] 10.222.0.23 () {60 vars in 1152 bytes} [Thu May 19 12:19:57 2022] POST /identity/v3/projects => generated 389 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 12:19:58.016095 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4ab36a-c900-4e99-abcf-ea911cb62d1e admin admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:19:58.079420 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-4adc1c34-8e00-48e5-96f7-dfdfc5873655 admin admin] The token cache is being invalidate because project 3749085b2e8d4adfa3067ccdfbffd023 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80540) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 12:19:58.079717 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-4adc1c34-8e00-48e5-96f7-dfdfc5873655 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:19:58.080392 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-4adc1c34-8e00-48e5-96f7-dfdfc5873655 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:19:58.088942 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1062/2116] 10.222.0.23 () {58 vars in 1232 bytes} [Thu May 19 12:19:57 2022] DELETE /identity/v3/projects/3749085b2e8d4adfa3067ccdfbffd023 => generated 0 bytes in 436 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:19:58.088942 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4adc1c34-8e00-48e5-96f7-dfdfc5873655 admin admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:19:58.089132 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4ab36a-c900-4e99-abcf-ea911cb62d1e admin admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:19:58.090463 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-4adc1c34-8e00-48e5-96f7-dfdfc5873655 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:19:58.097012 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68fa3501-e7e9-45bd-be78-1e6db46da9e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:19:58.097688 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fa3501-e7e9-45bd-be78-1e6db46da9e4 admin admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:19:58.097968 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fa3501-e7e9-45bd-be78-1e6db46da9e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:19:58.098243 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fa3501-e7e9-45bd-be78-1e6db46da9e4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:19:58.098991 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68fa3501-e7e9-45bd-be78-1e6db46da9e4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:19:58.099300 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68fa3501-e7e9-45bd-be78-1e6db46da9e4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:19:58.100977 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68fa3501-e7e9-45bd-be78-1e6db46da9e4 admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:19:58.118668 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1055/2117] 10.222.0.23 () {58 vars in 1152 bytes} [Thu May 19 12:19:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:19:58.118668 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5aee26-c7ed-4bee-a7a7-29a381dc1851 None None] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:19:58.118668 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5aee26-c7ed-4bee-a7a7-29a381dc1851 None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:19:58.118668 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5aee26-c7ed-4bee-a7a7-29a381dc1851 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:19:58.137947 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.auth.core [None req-0e5aee26-c7ed-4bee-a7a7-29a381dc1851 None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:19:58.139242 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-0e5aee26-c7ed-4bee-a7a7-29a381dc1851 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:19:58.147709 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1056/2118] 10.222.0.23 () {58 vars in 957 bytes} [Thu May 19 12:19:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:19:58.149335 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5aee26-c7ed-4bee-a7a7-29a381dc1851 None None] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:19:58.155688 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-0e5aee26-c7ed-4bee-a7a7-29a381dc1851 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:19:58.170004 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5aee26-c7ed-4bee-a7a7-29a381dc1851 None None] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:19:58.175876 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4adc1c34-8e00-48e5-96f7-dfdfc5873655 admin admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:19:58.176795 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0cacb5-7b17-4d74-a2b9-14abf35f368f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:19:58.177555 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0cacb5-7b17-4d74-a2b9-14abf35f368f admin admin] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:19:58.178025 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0cacb5-7b17-4d74-a2b9-14abf35f368f admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:19:58.178363 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0cacb5-7b17-4d74-a2b9-14abf35f368f admin admin] PATH_INFO: `/v3/projects` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:19:58.179721 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0cacb5-7b17-4d74-a2b9-14abf35f368f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:19:58.182216 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0cacb5-7b17-4d74-a2b9-14abf35f368f admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:19:58.192829 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0573d5dd-66ea-4c82-836c-71ba3fc92690 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:19:58.197965 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0573d5dd-66ea-4c82-836c-71ba3fc92690 admin admin] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:19:58.197965 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0573d5dd-66ea-4c82-836c-71ba3fc92690 admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:19:58.197965 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0573d5dd-66ea-4c82-836c-71ba3fc92690 admin admin] PATH_INFO: `/v3/users` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:19:58.197965 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0573d5dd-66ea-4c82-836c-71ba3fc92690 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:19:58.200431 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0573d5dd-66ea-4c82-836c-71ba3fc92690 admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:19:58.348629 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1063/2119] 10.222.0.23 () {60 vars in 1143 bytes} [Thu May 19 12:19:58 2022] POST /identity/v3/users => generated 295 bytes in 265 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 12:19:58.349884 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adc2efe-00c2-4e8a-a700-d6328d4df7f8 None None] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:19:58.350181 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adc2efe-00c2-4e8a-a700-d6328d4df7f8 None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:19:58.353526 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adc2efe-00c2-4e8a-a700-d6328d4df7f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:19:58.353679 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1057/2120] 10.222.0.23 () {60 vars in 1152 bytes} [Thu May 19 12:19:58 2022] POST /identity/v3/projects => generated 367 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 12:19:58.353679 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0cacb5-7b17-4d74-a2b9-14abf35f368f admin admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:19:58.355131 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-4c0cacb5-7b17-4d74-a2b9-14abf35f368f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:19:58.373136 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.auth.core [None req-7adc2efe-00c2-4e8a-a700-d6328d4df7f8 None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:19:58.374476 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-7adc2efe-00c2-4e8a-a700-d6328d4df7f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:19:58.439535 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0cacb5-7b17-4d74-a2b9-14abf35f368f admin admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:19:58.446505 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ad639f-1a80-4a5c-b2c3-d120a53a5d09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:19:58.447334 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1064/2121] 10.222.0.23 () {58 vars in 957 bytes} [Thu May 19 12:19:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:19:58.447696 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ad639f-1a80-4a5c-b2c3-d120a53a5d09 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:19:58.448009 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ad639f-1a80-4a5c-b2c3-d120a53a5d09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:19:58.448291 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ad639f-1a80-4a5c-b2c3-d120a53a5d09 admin admin] PATH_INFO: `/v3/projects/c7356f68a0aa43cfade192f26082f619` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:19:58.448446 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7adc2efe-00c2-4e8a-a700-d6328d4df7f8 None None] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:19:58.449571 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ad639f-1a80-4a5c-b2c3-d120a53a5d09 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c7356f68a0aa43cfade192f26082f619)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:19:58.451684 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ad639f-1a80-4a5c-b2c3-d120a53a5d09 admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:19:58.460442 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7adc2efe-00c2-4e8a-a700-d6328d4df7f8 None None] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:19:58.468779 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b00dcc-656b-4733-b507-9ca58614f2a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:19:58.469549 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b00dcc-656b-4733-b507-9ca58614f2a7 admin admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:19:58.470061 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b00dcc-656b-4733-b507-9ca58614f2a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:19:58.470410 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b00dcc-656b-4733-b507-9ca58614f2a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:19:58.471563 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b00dcc-656b-4733-b507-9ca58614f2a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:19:58.473656 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b00dcc-656b-4733-b507-9ca58614f2a7 admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:19:58.488688 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1065/2122] 10.222.0.23 () {58 vars in 1121 bytes} [Thu May 19 12:19:58 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 12:19:58.489843 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b00dcc-656b-4733-b507-9ca58614f2a7 admin admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:19:58.503010 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b00dcc-656b-4733-b507-9ca58614f2a7 admin admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:19:58.508139 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf03196-402c-4453-8041-bb6441d03354 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:19:58.508833 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf03196-402c-4453-8041-bb6441d03354 admin admin] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:19:58.512020 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf03196-402c-4453-8041-bb6441d03354 admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:19:58.512020 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf03196-402c-4453-8041-bb6441d03354 admin admin] PATH_INFO: `/v3/users` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:19:58.512020 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf03196-402c-4453-8041-bb6441d03354 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:19:58.512020 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf03196-402c-4453-8041-bb6441d03354 admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:19:58.580035 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-99ad639f-1a80-4a5c-b2c3-d120a53a5d09 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c7356f68a0aa43cfade192f26082f619', 'request_id': 'req-99ad639f-1a80-4a5c-b2c3-d120a53a5d09'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:19:58.599442 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1066/2123] 10.222.0.23 () {60 vars in 1143 bytes} [Thu May 19 12:19:58 2022] POST /identity/v3/users => generated 284 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 12:19:58.599442 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf03196-402c-4453-8041-bb6441d03354 admin admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:19:58.599682 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-99ad639f-1a80-4a5c-b2c3-d120a53a5d09 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c7356f68a0aa43cfade192f26082f619', 'request_id': 'req-99ad639f-1a80-4a5c-b2c3-d120a53a5d09'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:19:58.610567 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf03196-402c-4453-8041-bb6441d03354 admin admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:19:58.672462 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0303d9-3936-4781-a038-c53b50c1af24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:19:58.673139 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0303d9-3936-4781-a038-c53b50c1af24 admin admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:19:58.673429 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0303d9-3936-4781-a038-c53b50c1af24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:19:58.673849 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0303d9-3936-4781-a038-c53b50c1af24 admin admin] PATH_INFO: `/v3/domains` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:19:58.674594 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0303d9-3936-4781-a038-c53b50c1af24 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:19:58.674902 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0303d9-3936-4781-a038-c53b50c1af24 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:19:58.676562 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0303d9-3936-4781-a038-c53b50c1af24 admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:19:58.689444 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1067/2124] 10.222.0.23 () {58 vars in 1152 bytes} [Thu May 19 12:19:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:19:58.690345 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0303d9-3936-4781-a038-c53b50c1af24 admin admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:19:58.700565 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0303d9-3936-4781-a038-c53b50c1af24 admin admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:19:58.705307 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c095b7-2595-46ac-bbe7-d837eb2feff3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:19:58.705957 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c095b7-2595-46ac-bbe7-d837eb2feff3 admin admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:19:58.706255 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c095b7-2595-46ac-bbe7-d837eb2feff3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:19:58.706528 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c095b7-2595-46ac-bbe7-d837eb2feff3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:19:58.707242 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c095b7-2595-46ac-bbe7-d837eb2feff3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:19:58.708885 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c095b7-2595-46ac-bbe7-d837eb2feff3 admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:19:58.713871 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-99ad639f-1a80-4a5c-b2c3-d120a53a5d09 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c7356f68a0aa43cfade192f26082f619', 'request_id': 'req-99ad639f-1a80-4a5c-b2c3-d120a53a5d09'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:19:58.723550 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1068/2125] 10.222.0.23 () {58 vars in 1121 bytes} [Thu May 19 12:19:58 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 12:19:58.723550 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c095b7-2595-46ac-bbe7-d837eb2feff3 admin admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:19:58.727504 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-99ad639f-1a80-4a5c-b2c3-d120a53a5d09 admin admin] The token cache is being invalidate because project c7356f68a0aa43cfade192f26082f619 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80539) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 12:19:58.727861 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-99ad639f-1a80-4a5c-b2c3-d120a53a5d09 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:19:58.729408 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-99ad639f-1a80-4a5c-b2c3-d120a53a5d09 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:19:58.731409 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1058/2126] 10.222.0.23 () {58 vars in 1232 bytes} [Thu May 19 12:19:58 2022] DELETE /identity/v3/projects/c7356f68a0aa43cfade192f26082f619 => generated 0 bytes in 380 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:19:58.732452 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ad639f-1a80-4a5c-b2c3-d120a53a5d09 admin admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:19:58.734095 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-99ad639f-1a80-4a5c-b2c3-d120a53a5d09 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:19:58.736090 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c095b7-2595-46ac-bbe7-d837eb2feff3 admin admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:19:58.741471 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6230fb-1a8d-46ba-b806-1c99cc12a2ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:19:58.743145 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6230fb-1a8d-46ba-b806-1c99cc12a2ee admin admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:19:58.743744 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6230fb-1a8d-46ba-b806-1c99cc12a2ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:19:58.744200 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6230fb-1a8d-46ba-b806-1c99cc12a2ee admin admin] PATH_INFO: `/v3/roles` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:19:58.745668 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6230fb-1a8d-46ba-b806-1c99cc12a2ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:19:58.748112 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6230fb-1a8d-46ba-b806-1c99cc12a2ee admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:19:58.750252 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:19:58.753293 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ad639f-1a80-4a5c-b2c3-d120a53a5d09 admin admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:19:58.759059 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75f606e-561d-4637-9a43-21de8209379c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:19:58.759995 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1069/2127] 10.222.0.23 () {58 vars in 1121 bytes} [Thu May 19 12:19:58 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 12:19:58.760144 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75f606e-561d-4637-9a43-21de8209379c admin admin] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:19:58.760392 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75f606e-561d-4637-9a43-21de8209379c admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:19:58.760676 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6230fb-1a8d-46ba-b806-1c99cc12a2ee admin admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:19:58.760829 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75f606e-561d-4637-9a43-21de8209379c admin admin] PATH_INFO: `/v3/projects` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:19:58.761601 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75f606e-561d-4637-9a43-21de8209379c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:19:58.763087 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-1a6230fb-1a8d-46ba-b806-1c99cc12a2ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:19:58.764063 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75f606e-561d-4637-9a43-21de8209379c admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:19:58.778153 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6230fb-1a8d-46ba-b806-1c99cc12a2ee admin admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:19:58.783379 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8f8d02-e707-4556-aefb-e079742aa1b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:19:58.784320 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8f8d02-e707-4556-aefb-e079742aa1b9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:19:58.787535 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8f8d02-e707-4556-aefb-e079742aa1b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:19:58.787535 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8f8d02-e707-4556-aefb-e079742aa1b9 admin admin] PATH_INFO: `/v3/projects/1952e5f71bb245de912a1249fd7f6167/users/3cdc4cc4d83b4495aa44de70147a001c/roles/259eb400cdb4439db72d14456ae9cd08` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:19:58.801498 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8f8d02-e707-4556-aefb-e079742aa1b9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1952e5f71bb245de912a1249fd7f6167, user_id=3cdc4cc4d83b4495aa44de70147a001c, role_id=259eb400cdb4439db72d14456ae9cd08)` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:19:58.804624 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8f8d02-e707-4556-aefb-e079742aa1b9 admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:19:59.020130 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1059/2128] 10.222.0.23 () {60 vars in 1152 bytes} [Thu May 19 12:19:58 2022] POST /identity/v3/projects => generated 377 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 12:19:59.020130 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75f606e-561d-4637-9a43-21de8209379c admin admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:19:59.020130 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-e75f606e-561d-4637-9a43-21de8209379c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:19:59.045439 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75f606e-561d-4637-9a43-21de8209379c admin admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:19:59.050637 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20c47a1-dd5f-4a8e-9d08-41eeee6a06d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:19:59.051547 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20c47a1-dd5f-4a8e-9d08-41eeee6a06d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:19:59.051867 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20c47a1-dd5f-4a8e-9d08-41eeee6a06d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:19:59.052155 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20c47a1-dd5f-4a8e-9d08-41eeee6a06d0 admin admin] PATH_INFO: `/v3/users/ac09aa384cd8426ba338b618e765c1ca` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:19:59.053481 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20c47a1-dd5f-4a8e-9d08-41eeee6a06d0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ac09aa384cd8426ba338b618e765c1ca)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:19:59.056124 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20c47a1-dd5f-4a8e-9d08-41eeee6a06d0 admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:19:59.126734 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1070/2129] 10.222.0.23 () {60 vars in 1482 bytes} [Thu May 19 12:19:58 2022] PUT /identity/v3/projects/1952e5f71bb245de912a1249fd7f6167/users/3cdc4cc4d83b4495aa44de70147a001c/roles/259eb400cdb4439db72d14456ae9cd08 => generated 0 bytes in 366 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:19:59.127708 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8f8d02-e707-4556-aefb-e079742aa1b9 admin admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:19:59.129419 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-0e8f8d02-e707-4556-aefb-e079742aa1b9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:19:59.207159 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8f8d02-e707-4556-aefb-e079742aa1b9 admin admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:19:59.213235 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3fb550-d716-4f26-ad58-b615d19b3993 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:19:59.213755 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3fb550-d716-4f26-ad58-b615d19b3993 admin admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:19:59.214044 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3fb550-d716-4f26-ad58-b615d19b3993 admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:19:59.214324 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3fb550-d716-4f26-ad58-b615d19b3993 admin admin] PATH_INFO: `/v3/roles` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:19:59.215067 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3fb550-d716-4f26-ad58-b615d19b3993 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:19:59.216755 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3fb550-d716-4f26-ad58-b615d19b3993 admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:19:59.228472 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1071/2130] 10.222.0.23 () {58 vars in 1121 bytes} [Thu May 19 12:19:59 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 12:19:59.229320 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3fb550-d716-4f26-ad58-b615d19b3993 admin admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:19:59.243306 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3fb550-d716-4f26-ad58-b615d19b3993 admin admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:19:59.248661 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e9cce6-2526-446c-b9a2-d0c9d2bd1fc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:19:59.249332 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e9cce6-2526-446c-b9a2-d0c9d2bd1fc8 admin admin] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:19:59.249682 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e9cce6-2526-446c-b9a2-d0c9d2bd1fc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:19:59.249985 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e9cce6-2526-446c-b9a2-d0c9d2bd1fc8 admin admin] PATH_INFO: `/v3/users` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:19:59.250991 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e9cce6-2526-446c-b9a2-d0c9d2bd1fc8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:19:59.253875 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e9cce6-2526-446c-b9a2-d0c9d2bd1fc8 admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:19:59.369996 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-a20c47a1-dd5f-4a8e-9d08-41eeee6a06d0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ac09aa384cd8426ba338b618e765c1ca'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:19:59.382351 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-a20c47a1-dd5f-4a8e-9d08-41eeee6a06d0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ac09aa384cd8426ba338b618e765c1ca'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:19:59.383701 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-a20c47a1-dd5f-4a8e-9d08-41eeee6a06d0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ac09aa384cd8426ba338b618e765c1ca'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:19:59.394900 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1072/2131] 10.222.0.23 () {60 vars in 1143 bytes} [Thu May 19 12:19:59 2022] POST /identity/v3/users => generated 289 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 12:19:59.396449 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9d18a9-ce71-4160-89d9-ec99b6542de0 None None] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:19:59.396743 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9d18a9-ce71-4160-89d9-ec99b6542de0 None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:19:59.397027 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9d18a9-ce71-4160-89d9-ec99b6542de0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:19:59.417273 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.auth.core [None req-9a9d18a9-ce71-4160-89d9-ec99b6542de0 None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:19:59.418481 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-9a9d18a9-ce71-4160-89d9-ec99b6542de0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:19:59.426207 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1073/2132] 10.222.0.23 () {58 vars in 957 bytes} [Thu May 19 12:19:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:19:59.426972 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9d18a9-ce71-4160-89d9-ec99b6542de0 None None] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:19:59.437548 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9d18a9-ce71-4160-89d9-ec99b6542de0 None None] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:19:59.442185 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f981bdc1-2093-4c74-a0ae-690403498c68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:19:59.443052 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f981bdc1-2093-4c74-a0ae-690403498c68 admin admin] REQUEST_METHOD: `PUT` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:19:59.443335 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f981bdc1-2093-4c74-a0ae-690403498c68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:19:59.443611 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f981bdc1-2093-4c74-a0ae-690403498c68 admin admin] PATH_INFO: `/v3/projects/421d67219e5f47218f4f630ce4546ae1/users/61e41bdc63cb446b85bafc442b16ce23/roles/259eb400cdb4439db72d14456ae9cd08` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:19:59.460619 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f981bdc1-2093-4c74-a0ae-690403498c68 admin admin] RBAC: Authorizing `identity:create_grant(project_id=421d67219e5f47218f4f630ce4546ae1, user_id=61e41bdc63cb446b85bafc442b16ce23, role_id=259eb400cdb4439db72d14456ae9cd08)` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:19:59.463535 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f981bdc1-2093-4c74-a0ae-690403498c68 admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:19:59.502178 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-a20c47a1-dd5f-4a8e-9d08-41eeee6a06d0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ac09aa384cd8426ba338b618e765c1ca'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:19:59.509716 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-a20c47a1-dd5f-4a8e-9d08-41eeee6a06d0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ac09aa384cd8426ba338b618e765c1ca'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:19:59.515869 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1060/2133] 10.222.0.23 () {58 vars in 1223 bytes} [Thu May 19 12:19:58 2022] DELETE /identity/v3/users/ac09aa384cd8426ba338b618e765c1ca => generated 0 bytes in 563 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:19:59.516657 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20c47a1-dd5f-4a8e-9d08-41eeee6a06d0 admin admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:19:59.517966 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-a20c47a1-dd5f-4a8e-9d08-41eeee6a06d0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:19:59.577727 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1074/2134] 10.222.0.23 () {60 vars in 1482 bytes} [Thu May 19 12:19:59 2022] PUT /identity/v3/projects/421d67219e5f47218f4f630ce4546ae1/users/61e41bdc63cb446b85bafc442b16ce23/roles/259eb400cdb4439db72d14456ae9cd08 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:19:59.578717 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f981bdc1-2093-4c74-a0ae-690403498c68 admin admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:19:59.580765 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-f981bdc1-2093-4c74-a0ae-690403498c68 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:19:59.595984 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20c47a1-dd5f-4a8e-9d08-41eeee6a06d0 admin admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:19:59.647654 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f981bdc1-2093-4c74-a0ae-690403498c68 admin admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:19:59.653879 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace75c15-e687-4057-8158-7206dbed6161 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a00b3fa7497456895e5e0bf105c4fc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd70d25bf69314820b8aaecb65576978a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:19:59.654545 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace75c15-e687-4057-8158-7206dbed6161 service neutron] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:19:59.654836 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace75c15-e687-4057-8158-7206dbed6161 service neutron] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:19:59.655123 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace75c15-e687-4057-8158-7206dbed6161 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:19:59.657139 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-ace75c15-e687-4057-8158-7206dbed6161 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:19:59.662097 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b24179-1bcf-482f-b908-8c6534f9175a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:19:59.662951 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b24179-1bcf-482f-b908-8c6534f9175a admin admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:19:59.663166 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b24179-1bcf-482f-b908-8c6534f9175a admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:19:59.663639 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b24179-1bcf-482f-b908-8c6534f9175a admin admin] PATH_INFO: `/v3/roles` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:19:59.664948 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b24179-1bcf-482f-b908-8c6534f9175a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:19:59.666848 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b24179-1bcf-482f-b908-8c6534f9175a admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:19:59.670362 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace75c15-e687-4057-8158-7206dbed6161 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:19:59.671832 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace75c15-e687-4057-8158-7206dbed6161 service neutron] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:19:59.677109 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1061/2135] 10.222.0.23 () {58 vars in 1121 bytes} [Thu May 19 12:19:59 2022] GET /identity/v3/roles => generated 1420 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 12:19:59.678497 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7dda56b-b4d0-41f8-8ed4-b320b7f2b0e5 None None] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:19:59.678788 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7dda56b-b4d0-41f8-8ed4-b320b7f2b0e5 None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:19:59.679064 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7dda56b-b4d0-41f8-8ed4-b320b7f2b0e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:19:59.684507 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1075/2136] 10.222.0.23 () {60 vars in 1365 bytes} [Thu May 19 12:19:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:19:59.686044 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace75c15-e687-4057-8158-7206dbed6161 service neutron] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:19:59.695659 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace75c15-e687-4057-8158-7206dbed6161 service neutron] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:19:59.701507 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02556efb-4bef-4ac2-8e7b-1f79c9e37c70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:19:59.702185 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02556efb-4bef-4ac2-8e7b-1f79c9e37c70 admin admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:19:59.702465 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02556efb-4bef-4ac2-8e7b-1f79c9e37c70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:19:59.703072 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02556efb-4bef-4ac2-8e7b-1f79c9e37c70 admin admin] PATH_INFO: `/v3/roles` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:19:59.703482 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02556efb-4bef-4ac2-8e7b-1f79c9e37c70 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:19:59.705165 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02556efb-4bef-4ac2-8e7b-1f79c9e37c70 admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:19:59.709977 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.auth.core [None req-c7dda56b-b4d0-41f8-8ed4-b320b7f2b0e5 None None] MFA Rules not processed for user `61e41bdc63cb446b85bafc442b16ce23`. Rule list: `[]` (Enabled: `True`). {{(pid=80539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:19:59.712001 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-c7dda56b-b4d0-41f8-8ed4-b320b7f2b0e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:19:59.721092 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1076/2137] 10.222.0.23 () {58 vars in 1121 bytes} [Thu May 19 12:19:59 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 12:19:59.724200 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02556efb-4bef-4ac2-8e7b-1f79c9e37c70 admin admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:19:59.736419 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02556efb-4bef-4ac2-8e7b-1f79c9e37c70 admin admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:19:59.742855 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1e666e-4fd9-41a7-a302-9fe6454ec96c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:19:59.746480 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1e666e-4fd9-41a7-a302-9fe6454ec96c admin admin] REQUEST_METHOD: `PUT` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:19:59.746480 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1e666e-4fd9-41a7-a302-9fe6454ec96c admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:19:59.746480 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1e666e-4fd9-41a7-a302-9fe6454ec96c admin admin] PATH_INFO: `/v3/projects/ca857a614c1745df8b8a1f91a46a6394/users/45c84d2b727f47e380c6a78423f170ff/roles/259eb400cdb4439db72d14456ae9cd08` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:19:59.763066 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1e666e-4fd9-41a7-a302-9fe6454ec96c admin admin] RBAC: Authorizing `identity:create_grant(project_id=ca857a614c1745df8b8a1f91a46a6394, user_id=45c84d2b727f47e380c6a78423f170ff, role_id=259eb400cdb4439db72d14456ae9cd08)` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:19:59.766190 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1e666e-4fd9-41a7-a302-9fe6454ec96c admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:19:59.800420 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1062/2138] 10.222.0.23 () {58 vars in 957 bytes} [Thu May 19 12:19:59 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:19:59.812199 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7dda56b-b4d0-41f8-8ed4-b320b7f2b0e5 None None] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:19:59.815327 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-c7dda56b-b4d0-41f8-8ed4-b320b7f2b0e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:19:59.842304 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1077/2139] 10.222.0.23 () {60 vars in 1482 bytes} [Thu May 19 12:19:59 2022] PUT /identity/v3/projects/ca857a614c1745df8b8a1f91a46a6394/users/45c84d2b727f47e380c6a78423f170ff/roles/259eb400cdb4439db72d14456ae9cd08 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:19:59.853990 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e679715d-9ebd-4c29-8720-6b4160e6416f None None] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:19:59.854272 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e679715d-9ebd-4c29-8720-6b4160e6416f None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:19:59.854815 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e679715d-9ebd-4c29-8720-6b4160e6416f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:19:59.881587 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7dda56b-b4d0-41f8-8ed4-b320b7f2b0e5 None None] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:19:59.883834 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.auth.core [None req-e679715d-9ebd-4c29-8720-6b4160e6416f None None] MFA Rules not processed for user `45c84d2b727f47e380c6a78423f170ff`. Rule list: `[]` (Enabled: `True`). {{(pid=80540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:19:59.885086 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-e679715d-9ebd-4c29-8720-6b4160e6416f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:19:59.943151 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac9d4fe-51b5-451c-bffa-80c2355c8372 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8800254c0e784e42a791fb9abc274d79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd70d25bf69314820b8aaecb65576978a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:19:59.944081 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac9d4fe-51b5-451c-bffa-80c2355c8372 service cinder] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:19:59.944081 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac9d4fe-51b5-451c-bffa-80c2355c8372 service cinder] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:19:59.944287 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac9d4fe-51b5-451c-bffa-80c2355c8372 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:19:59.970861 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1078/2140] 10.222.0.23 () {58 vars in 957 bytes} [Thu May 19 12:19:59 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 12:19:59.980590 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e679715d-9ebd-4c29-8720-6b4160e6416f None None] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:20:00.011042 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac9d4fe-51b5-451c-bffa-80c2355c8372 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:20:00.012529 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac9d4fe-51b5-451c-bffa-80c2355c8372 service cinder] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:20:00.022485 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1063/2141] 10.222.0.23 () {60 vars in 1365 bytes} [Thu May 19 12:19:59 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 211 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 12:20:00.044649 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac9d4fe-51b5-451c-bffa-80c2355c8372 service cinder] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:20:00.049694 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-3ac9d4fe-51b5-451c-bffa-80c2355c8372 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:20:00.053902 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e679715d-9ebd-4c29-8720-6b4160e6416f None None] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:20:00.060199 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c08119-9e9a-403c-9b6e-1589b8a0e1ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:20:00.061318 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c08119-9e9a-403c-9b6e-1589b8a0e1ce admin admin] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:20:00.061695 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac9d4fe-51b5-451c-bffa-80c2355c8372 service cinder] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:20:00.062219 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c08119-9e9a-403c-9b6e-1589b8a0e1ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:20:00.063177 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c08119-9e9a-403c-9b6e-1589b8a0e1ce admin admin] PATH_INFO: `/v3/projects` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:20:00.066016 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=89,keystone:DELETE=9,keystone:INSERT=2 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:20:00.067203 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c08119-9e9a-403c-9b6e-1589b8a0e1ce admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:20:00.067599 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9f8321-99e3-449b-bea3-6477e0f7e157 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:20:00.068564 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9f8321-99e3-449b-bea3-6477e0f7e157 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:20:00.074656 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9f8321-99e3-449b-bea3-6477e0f7e157 admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:20:00.074656 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9f8321-99e3-449b-bea3-6477e0f7e157 admin admin] PATH_INFO: `/v3/users/61e41bdc63cb446b85bafc442b16ce23` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:20:00.074981 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c08119-9e9a-403c-9b6e-1589b8a0e1ce admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:20:00.075062 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9f8321-99e3-449b-bea3-6477e0f7e157 admin admin] RBAC: Authorizing `identity:delete_user(user_id=61e41bdc63cb446b85bafc442b16ce23)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:20:00.075062 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9f8321-99e3-449b-bea3-6477e0f7e157 admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:20:00.196434 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1079/2142] 10.222.0.23 () {60 vars in 1152 bytes} [Thu May 19 12:19:59 2022] POST /identity/v3/projects => generated 377 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 12:20:00.197586 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c08119-9e9a-403c-9b6e-1589b8a0e1ce admin admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:20:00.199435 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-41c08119-9e9a-403c-9b6e-1589b8a0e1ce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:20:00.262194 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-3e9f8321-99e3-449b-bea3-6477e0f7e157 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '61e41bdc63cb446b85bafc442b16ce23'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:20:00.288327 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c08119-9e9a-403c-9b6e-1589b8a0e1ce admin admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:20:00.297651 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-3e9f8321-99e3-449b-bea3-6477e0f7e157 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '61e41bdc63cb446b85bafc442b16ce23'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:20:00.298874 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53af9a1f-c01d-4f36-bac6-1b9212b1b84a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:20:00.299015 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-3e9f8321-99e3-449b-bea3-6477e0f7e157 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '61e41bdc63cb446b85bafc442b16ce23'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:20:00.300186 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53af9a1f-c01d-4f36-bac6-1b9212b1b84a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:20:00.301426 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53af9a1f-c01d-4f36-bac6-1b9212b1b84a admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:20:00.302072 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53af9a1f-c01d-4f36-bac6-1b9212b1b84a admin admin] PATH_INFO: `/v3/projects/adc7c7b781734864a8f2977b38b85479` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:20:00.303680 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53af9a1f-c01d-4f36-bac6-1b9212b1b84a admin admin] RBAC: Authorizing `identity:delete_project(project_id=adc7c7b781734864a8f2977b38b85479)` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:20:00.308912 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53af9a1f-c01d-4f36-bac6-1b9212b1b84a admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:20:00.411297 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-3e9f8321-99e3-449b-bea3-6477e0f7e157 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '61e41bdc63cb446b85bafc442b16ce23'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:20:00.423479 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-3e9f8321-99e3-449b-bea3-6477e0f7e157 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '61e41bdc63cb446b85bafc442b16ce23'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:20:00.451142 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1064/2143] 10.222.0.23 () {58 vars in 1223 bytes} [Thu May 19 12:20:00 2022] DELETE /identity/v3/users/61e41bdc63cb446b85bafc442b16ce23 => generated 0 bytes in 407 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:20:00.452210 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9f8321-99e3-449b-bea3-6477e0f7e157 admin admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:20:00.454297 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-3e9f8321-99e3-449b-bea3-6477e0f7e157 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:20:00.462734 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-53af9a1f-c01d-4f36-bac6-1b9212b1b84a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'adc7c7b781734864a8f2977b38b85479', 'request_id': 'req-53af9a1f-c01d-4f36-bac6-1b9212b1b84a'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:20:00.471017 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-53af9a1f-c01d-4f36-bac6-1b9212b1b84a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'adc7c7b781734864a8f2977b38b85479', 'request_id': 'req-53af9a1f-c01d-4f36-bac6-1b9212b1b84a'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:20:00.477612 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-53af9a1f-c01d-4f36-bac6-1b9212b1b84a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'adc7c7b781734864a8f2977b38b85479', 'request_id': 'req-53af9a1f-c01d-4f36-bac6-1b9212b1b84a'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:20:00.528902 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9f8321-99e3-449b-bea3-6477e0f7e157 admin admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:20:00.533883 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fefe4be9-9cc6-41b7-8084-8263d75a6147 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:20:00.538598 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefe4be9-9cc6-41b7-8084-8263d75a6147 admin admin] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:20:00.538598 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefe4be9-9cc6-41b7-8084-8263d75a6147 admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:20:00.538598 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefe4be9-9cc6-41b7-8084-8263d75a6147 admin admin] PATH_INFO: `/v3/users` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:20:00.538598 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fefe4be9-9cc6-41b7-8084-8263d75a6147 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:20:00.538598 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fefe4be9-9cc6-41b7-8084-8263d75a6147 admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:20:00.565200 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-53af9a1f-c01d-4f36-bac6-1b9212b1b84a admin admin] The token cache is being invalidate because project adc7c7b781734864a8f2977b38b85479 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80540) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 12:20:00.565499 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-53af9a1f-c01d-4f36-bac6-1b9212b1b84a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:20:00.566177 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-53af9a1f-c01d-4f36-bac6-1b9212b1b84a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:20:00.567850 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1080/2144] 10.222.0.23 () {58 vars in 1232 bytes} [Thu May 19 12:20:00 2022] DELETE /identity/v3/projects/adc7c7b781734864a8f2977b38b85479 => generated 0 bytes in 371 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:20:00.569198 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a57fc21-0647-4d21-bb9a-36efd60392c0 None None] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:20:00.569518 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a57fc21-0647-4d21-bb9a-36efd60392c0 None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:20:00.570137 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a57fc21-0647-4d21-bb9a-36efd60392c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:20:00.599372 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.auth.core [None req-2a57fc21-0647-4d21-bb9a-36efd60392c0 None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:20:00.600711 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-2a57fc21-0647-4d21-bb9a-36efd60392c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:20:00.614782 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1081/2145] 10.222.0.23 () {58 vars in 957 bytes} [Thu May 19 12:20:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:20:00.628255 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a57fc21-0647-4d21-bb9a-36efd60392c0 None None] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:20:00.631463 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-2a57fc21-0647-4d21-bb9a-36efd60392c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:20:00.674406 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1065/2146] 10.222.0.23 () {60 vars in 1143 bytes} [Thu May 19 12:20:00 2022] POST /identity/v3/users => generated 289 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 12:20:00.675744 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fefe4be9-9cc6-41b7-8084-8263d75a6147 admin admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:20:00.679000 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-fefe4be9-9cc6-41b7-8084-8263d75a6147 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:20:00.702318 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fefe4be9-9cc6-41b7-8084-8263d75a6147 admin admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:20:00.709493 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f4927e-e69f-4af4-8713-3eee5c9d781b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:20:00.710227 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f4927e-e69f-4af4-8713-3eee5c9d781b admin admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:20:00.711963 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f4927e-e69f-4af4-8713-3eee5c9d781b admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:20:00.712276 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f4927e-e69f-4af4-8713-3eee5c9d781b admin admin] PATH_INFO: `/v3/roles` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:20:00.713075 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f4927e-e69f-4af4-8713-3eee5c9d781b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:20:00.715244 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a57fc21-0647-4d21-bb9a-36efd60392c0 None None] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:20:00.715736 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f4927e-e69f-4af4-8713-3eee5c9d781b admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:20:00.720560 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80168895-9e59-498f-b17b-9f67dea5d7ea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a00b3fa7497456895e5e0bf105c4fc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd70d25bf69314820b8aaecb65576978a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:20:00.721581 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80168895-9e59-498f-b17b-9f67dea5d7ea service neutron] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:20:00.722203 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80168895-9e59-498f-b17b-9f67dea5d7ea service neutron] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:20:00.722649 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80168895-9e59-498f-b17b-9f67dea5d7ea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:20:00.734801 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1066/2147] 10.222.0.23 () {58 vars in 1121 bytes} [Thu May 19 12:20:00 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 12:20:00.735997 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f4927e-e69f-4af4-8713-3eee5c9d781b admin admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:20:00.737376 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80168895-9e59-498f-b17b-9f67dea5d7ea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:20:00.740107 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80168895-9e59-498f-b17b-9f67dea5d7ea service neutron] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:20:00.748025 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f4927e-e69f-4af4-8713-3eee5c9d781b admin admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:20:00.753787 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1082/2148] 10.222.0.23 () {60 vars in 1365 bytes} [Thu May 19 12:20:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:20:00.759838 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5291f6ea-2479-4f78-9b68-a5070c89b22f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:20:00.760802 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5291f6ea-2479-4f78-9b68-a5070c89b22f admin admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:20:00.761182 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5291f6ea-2479-4f78-9b68-a5070c89b22f admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:20:00.761717 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5291f6ea-2479-4f78-9b68-a5070c89b22f admin admin] PATH_INFO: `/v3/roles` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:20:00.762679 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5291f6ea-2479-4f78-9b68-a5070c89b22f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:20:00.764582 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5291f6ea-2479-4f78-9b68-a5070c89b22f admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:20:00.779302 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1067/2149] 10.222.0.23 () {58 vars in 1121 bytes} [Thu May 19 12:20:00 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 12:20:00.787557 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80168895-9e59-498f-b17b-9f67dea5d7ea service neutron] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:20:00.799015 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80168895-9e59-498f-b17b-9f67dea5d7ea service neutron] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:20:00.803901 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160810d2-68de-4083-b2f5-28af8b1b7f50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:20:00.806086 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160810d2-68de-4083-b2f5-28af8b1b7f50 admin admin] REQUEST_METHOD: `PUT` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:20:00.806371 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160810d2-68de-4083-b2f5-28af8b1b7f50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:20:00.806659 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160810d2-68de-4083-b2f5-28af8b1b7f50 admin admin] PATH_INFO: `/v3/projects/3651442a6e6144fb99ae49c3db1107be/users/281e1bc4c0fc478ba98ee61f5b731c5a/roles/259eb400cdb4439db72d14456ae9cd08` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:20:00.823117 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160810d2-68de-4083-b2f5-28af8b1b7f50 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3651442a6e6144fb99ae49c3db1107be, user_id=281e1bc4c0fc478ba98ee61f5b731c5a, role_id=259eb400cdb4439db72d14456ae9cd08)` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:20:00.825992 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160810d2-68de-4083-b2f5-28af8b1b7f50 admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:20:00.955323 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1083/2150] 10.222.0.23 () {60 vars in 1482 bytes} [Thu May 19 12:20:00 2022] PUT /identity/v3/projects/3651442a6e6144fb99ae49c3db1107be/users/281e1bc4c0fc478ba98ee61f5b731c5a/roles/259eb400cdb4439db72d14456ae9cd08 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:20:00.967139 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3275a7-4b54-43ef-bcb3-6cd6c48a0dbc None None] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:20:00.967437 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3275a7-4b54-43ef-bcb3-6cd6c48a0dbc None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:20:00.967718 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3275a7-4b54-43ef-bcb3-6cd6c48a0dbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:20:00.999572 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.auth.core [None req-cd3275a7-4b54-43ef-bcb3-6cd6c48a0dbc None None] MFA Rules not processed for user `281e1bc4c0fc478ba98ee61f5b731c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:20:01.000874 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-cd3275a7-4b54-43ef-bcb3-6cd6c48a0dbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:20:01.074427 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1068/2151] 10.222.0.23 () {58 vars in 957 bytes} [Thu May 19 12:20:00 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:20:01.079951 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160810d2-68de-4083-b2f5-28af8b1b7f50 admin admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:20:01.160669 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160810d2-68de-4083-b2f5-28af8b1b7f50 admin admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:20:01.166245 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ced5f1-6c9d-4d25-acbd-c178c5c56a00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:20:01.166986 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ced5f1-6c9d-4d25-acbd-c178c5c56a00 admin admin] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:20:01.167222 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ced5f1-6c9d-4d25-acbd-c178c5c56a00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:20:01.167489 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ced5f1-6c9d-4d25-acbd-c178c5c56a00 admin admin] PATH_INFO: `/v3/projects` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:20:01.168237 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ced5f1-6c9d-4d25-acbd-c178c5c56a00 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:20:01.170191 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ced5f1-6c9d-4d25-acbd-c178c5c56a00 admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:20:01.407740 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3275a7-4b54-43ef-bcb3-6cd6c48a0dbc None None] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:20:01.409152 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-cd3275a7-4b54-43ef-bcb3-6cd6c48a0dbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:20:01.420041 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1084/2152] 10.222.0.23 () {60 vars in 1152 bytes} [Thu May 19 12:20:01 2022] POST /identity/v3/projects => generated 377 bytes in 341 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 12:20:01.421664 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ced5f1-6c9d-4d25-acbd-c178c5c56a00 admin admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:20:01.428537 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3275a7-4b54-43ef-bcb3-6cd6c48a0dbc None None] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:20:01.559304 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1688ab-c86f-4786-9a8f-5216382bb516 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:20:01.563991 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1688ab-c86f-4786-9a8f-5216382bb516 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:20:01.563991 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1688ab-c86f-4786-9a8f-5216382bb516 admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:20:01.563991 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1688ab-c86f-4786-9a8f-5216382bb516 admin admin] PATH_INFO: `/v3/projects/421d67219e5f47218f4f630ce4546ae1` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:20:01.575967 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=93,keystone:DELETE=5,keystone:INSERT=2 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:20:01.582065 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1688ab-c86f-4786-9a8f-5216382bb516 admin admin] RBAC: Authorizing `identity:delete_project(project_id=421d67219e5f47218f4f630ce4546ae1)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:20:01.586386 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1688ab-c86f-4786-9a8f-5216382bb516 admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:20:01.607959 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ced5f1-6c9d-4d25-acbd-c178c5c56a00 admin admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:20:01.618143 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cb39e9-7556-4b26-bb35-809f5ed92582 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:20:01.619428 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cb39e9-7556-4b26-bb35-809f5ed92582 admin admin] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:20:01.620104 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cb39e9-7556-4b26-bb35-809f5ed92582 admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:20:01.621755 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cb39e9-7556-4b26-bb35-809f5ed92582 admin admin] PATH_INFO: `/v3/users` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:20:01.623273 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13cb39e9-7556-4b26-bb35-809f5ed92582 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:20:01.627287 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13cb39e9-7556-4b26-bb35-809f5ed92582 admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:20:01.827750 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1085/2153] 10.222.0.23 () {60 vars in 1143 bytes} [Thu May 19 12:20:01 2022] POST /identity/v3/users => generated 295 bytes in 407 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 12:20:01.828522 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cb39e9-7556-4b26-bb35-809f5ed92582 admin admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:20:01.839318 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cb39e9-7556-4b26-bb35-809f5ed92582 admin admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:20:01.848583 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11adc1d-155a-41f5-b636-729d591775be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:20:01.849370 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11adc1d-155a-41f5-b636-729d591775be admin admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:20:01.849663 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11adc1d-155a-41f5-b636-729d591775be admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:20:01.849964 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11adc1d-155a-41f5-b636-729d591775be admin admin] PATH_INFO: `/v3/roles` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:20:01.850729 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11adc1d-155a-41f5-b636-729d591775be admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:20:01.852917 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11adc1d-155a-41f5-b636-729d591775be admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:20:01.865976 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1086/2154] 10.222.0.23 () {58 vars in 1121 bytes} [Thu May 19 12:20:01 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 12:20:01.872995 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11adc1d-155a-41f5-b636-729d591775be admin admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:20:01.883899 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11adc1d-155a-41f5-b636-729d591775be admin admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:20:01.888842 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c233e4be-e0f3-4d8c-9e90-54be2db89a73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:20:01.889782 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c233e4be-e0f3-4d8c-9e90-54be2db89a73 admin admin] REQUEST_METHOD: `PUT` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:20:01.890072 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c233e4be-e0f3-4d8c-9e90-54be2db89a73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:20:01.890372 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c233e4be-e0f3-4d8c-9e90-54be2db89a73 admin admin] PATH_INFO: `/v3/projects/60cd52fe4a574a418d586e6a38a38295/users/0d7e8c2c7ef448edbf173fe8ea50da77/roles/9db5671c65d3414ea2682afa2ccf1403` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:20:01.912408 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c233e4be-e0f3-4d8c-9e90-54be2db89a73 admin admin] RBAC: Authorizing `identity:create_grant(project_id=60cd52fe4a574a418d586e6a38a38295, user_id=0d7e8c2c7ef448edbf173fe8ea50da77, role_id=9db5671c65d3414ea2682afa2ccf1403)` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:20:01.913200 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c233e4be-e0f3-4d8c-9e90-54be2db89a73 admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:20:01.936071 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-ca1688ab-c86f-4786-9a8f-5216382bb516 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '421d67219e5f47218f4f630ce4546ae1', 'request_id': 'req-ca1688ab-c86f-4786-9a8f-5216382bb516'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:20:01.942996 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-ca1688ab-c86f-4786-9a8f-5216382bb516 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '421d67219e5f47218f4f630ce4546ae1', 'request_id': 'req-ca1688ab-c86f-4786-9a8f-5216382bb516'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:20:02.094582 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1087/2155] 10.222.0.23 () {60 vars in 1482 bytes} [Thu May 19 12:20:01 2022] PUT /identity/v3/projects/60cd52fe4a574a418d586e6a38a38295/users/0d7e8c2c7ef448edbf173fe8ea50da77/roles/9db5671c65d3414ea2682afa2ccf1403 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:20:02.096547 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-ca1688ab-c86f-4786-9a8f-5216382bb516 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '421d67219e5f47218f4f630ce4546ae1', 'request_id': 'req-ca1688ab-c86f-4786-9a8f-5216382bb516'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:20:02.099240 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c233e4be-e0f3-4d8c-9e90-54be2db89a73 admin admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:20:02.118982 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-ca1688ab-c86f-4786-9a8f-5216382bb516 admin admin] The token cache is being invalidate because project 421d67219e5f47218f4f630ce4546ae1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80539) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 12:20:02.119684 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-ca1688ab-c86f-4786-9a8f-5216382bb516 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:20:02.121171 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-ca1688ab-c86f-4786-9a8f-5216382bb516 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:20:02.122870 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1069/2156] 10.222.0.23 () {58 vars in 1232 bytes} [Thu May 19 12:20:01 2022] DELETE /identity/v3/projects/421d67219e5f47218f4f630ce4546ae1 => generated 0 bytes in 716 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 12:20:02.127847 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bf68a3-31e3-45d5-8013-8d176f678758 None None] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:20:02.128165 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bf68a3-31e3-45d5-8013-8d176f678758 None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:20:02.128449 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bf68a3-31e3-45d5-8013-8d176f678758 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:20:02.149050 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.auth.core [None req-b6bf68a3-31e3-45d5-8013-8d176f678758 None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:20:02.150387 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-b6bf68a3-31e3-45d5-8013-8d176f678758 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:20:02.177306 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c233e4be-e0f3-4d8c-9e90-54be2db89a73 admin admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:20:02.183893 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2f5d1b-4d8d-41ff-abaf-10dadf5be23d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:20:02.184870 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2f5d1b-4d8d-41ff-abaf-10dadf5be23d admin admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:20:02.185352 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2f5d1b-4d8d-41ff-abaf-10dadf5be23d admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:20:02.185690 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2f5d1b-4d8d-41ff-abaf-10dadf5be23d admin admin] PATH_INFO: `/v3/roles` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:20:02.186693 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2f5d1b-4d8d-41ff-abaf-10dadf5be23d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:20:02.188506 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2f5d1b-4d8d-41ff-abaf-10dadf5be23d admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:20:02.214679 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1088/2157] 10.222.0.23 () {58 vars in 1121 bytes} [Thu May 19 12:20:02 2022] GET /identity/v3/roles => generated 1420 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 12:20:02.216330 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2f5d1b-4d8d-41ff-abaf-10dadf5be23d admin admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:20:02.219192 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-5a2f5d1b-4d8d-41ff-abaf-10dadf5be23d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:20:02.234179 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1070/2158] 10.222.0.23 () {58 vars in 957 bytes} [Thu May 19 12:20:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 12:20:02.246789 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cadba9-4086-4bc1-92de-d834e89fe52b None None] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:20:02.247935 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cadba9-4086-4bc1-92de-d834e89fe52b None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:20:02.248451 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cadba9-4086-4bc1-92de-d834e89fe52b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:20:02.250305 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2f5d1b-4d8d-41ff-abaf-10dadf5be23d admin admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:20:02.269722 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae30e7f-b1ec-48ba-a3c0-a2b773cb9523 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:20:02.271351 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae30e7f-b1ec-48ba-a3c0-a2b773cb9523 admin admin] REQUEST_METHOD: `PUT` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:20:02.272153 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae30e7f-b1ec-48ba-a3c0-a2b773cb9523 admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:20:02.272649 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae30e7f-b1ec-48ba-a3c0-a2b773cb9523 admin admin] PATH_INFO: `/v3/projects/60cd52fe4a574a418d586e6a38a38295/users/0d7e8c2c7ef448edbf173fe8ea50da77/roles/9db5671c65d3414ea2682afa2ccf1403` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:20:02.275711 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae30e7f-b1ec-48ba-a3c0-a2b773cb9523 admin admin] RBAC: Authorizing `identity:create_grant(project_id=60cd52fe4a574a418d586e6a38a38295, user_id=0d7e8c2c7ef448edbf173fe8ea50da77, role_id=9db5671c65d3414ea2682afa2ccf1403)` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:20:02.279039 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae30e7f-b1ec-48ba-a3c0-a2b773cb9523 admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:20:02.280261 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.auth.core [None req-15cadba9-4086-4bc1-92de-d834e89fe52b None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:20:02.281501 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-15cadba9-4086-4bc1-92de-d834e89fe52b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:20:02.292630 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1071/2159] 10.222.0.23 () {58 vars in 957 bytes} [Thu May 19 12:20:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:20:02.294446 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1089/2160] 10.222.0.23 () {60 vars in 1482 bytes} [Thu May 19 12:20:02 2022] PUT /identity/v3/projects/60cd52fe4a574a418d586e6a38a38295/users/0d7e8c2c7ef448edbf173fe8ea50da77/roles/9db5671c65d3414ea2682afa2ccf1403 => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:20:02.305228 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15cadba9-4086-4bc1-92de-d834e89fe52b None None] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:20:02.308186 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b26c02-a235-4be2-9a42-875f56e01fd5 None None] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:20:02.308186 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b26c02-a235-4be2-9a42-875f56e01fd5 None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:20:02.308580 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b26c02-a235-4be2-9a42-875f56e01fd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:20:02.318959 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15cadba9-4086-4bc1-92de-d834e89fe52b None None] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:20:02.350989 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.auth.core [None req-c1b26c02-a235-4be2-9a42-875f56e01fd5 None None] MFA Rules not processed for user `0d7e8c2c7ef448edbf173fe8ea50da77`. Rule list: `[]` (Enabled: `True`). {{(pid=80540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:20:02.353259 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-c1b26c02-a235-4be2-9a42-875f56e01fd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:20:02.395913 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb51b0a3-c6f3-4925-b752-857f3bec53ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:20:02.397379 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb51b0a3-c6f3-4925-b752-857f3bec53ac admin admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:20:02.398636 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb51b0a3-c6f3-4925-b752-857f3bec53ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:20:02.399081 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb51b0a3-c6f3-4925-b752-857f3bec53ac admin admin] PATH_INFO: `/v3/domains` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:20:02.400371 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb51b0a3-c6f3-4925-b752-857f3bec53ac admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:20:02.400690 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb51b0a3-c6f3-4925-b752-857f3bec53ac admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:20:02.403640 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb51b0a3-c6f3-4925-b752-857f3bec53ac admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:20:02.418235 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1072/2161] 10.222.0.23 () {58 vars in 1152 bytes} [Thu May 19 12:20:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:20:02.423012 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb51b0a3-c6f3-4925-b752-857f3bec53ac admin admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:20:02.434184 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb51b0a3-c6f3-4925-b752-857f3bec53ac admin admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:20:02.444566 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae6f541-cc18-467c-bc12-2a4697ce9196 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:20:02.446508 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae6f541-cc18-467c-bc12-2a4697ce9196 admin admin] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:20:02.446972 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae6f541-cc18-467c-bc12-2a4697ce9196 admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:20:02.447412 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae6f541-cc18-467c-bc12-2a4697ce9196 admin admin] PATH_INFO: `/v3/projects` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:20:02.448300 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae6f541-cc18-467c-bc12-2a4697ce9196 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:20:02.450837 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae6f541-cc18-467c-bc12-2a4697ce9196 admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:20:02.456794 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1090/2162] 10.222.0.23 () {58 vars in 957 bytes} [Thu May 19 12:20:02 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:20:02.479487 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b26c02-a235-4be2-9a42-875f56e01fd5 None None] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:20:02.481405 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-c1b26c02-a235-4be2-9a42-875f56e01fd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:20:02.577250 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b26c02-a235-4be2-9a42-875f56e01fd5 None None] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:20:02.582399 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1073/2163] 10.222.0.23 () {60 vars in 1152 bytes} [Thu May 19 12:20:02 2022] POST /identity/v3/projects => generated 383 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 12:20:02.584290 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae6f541-cc18-467c-bc12-2a4697ce9196 admin admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:20:02.660344 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6996472b-a42a-4d1f-b3d8-e85047ed86ac service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8800254c0e784e42a791fb9abc274d79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd70d25bf69314820b8aaecb65576978a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:20:02.661107 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6996472b-a42a-4d1f-b3d8-e85047ed86ac service cinder] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:20:02.661307 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6996472b-a42a-4d1f-b3d8-e85047ed86ac service cinder] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:20:02.661596 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6996472b-a42a-4d1f-b3d8-e85047ed86ac service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:20:02.692811 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae6f541-cc18-467c-bc12-2a4697ce9196 admin admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:20:02.697493 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb7f48c-91df-432f-9929-42dc3ad33589 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:20:02.698192 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb7f48c-91df-432f-9929-42dc3ad33589 admin admin] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:20:02.698481 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb7f48c-91df-432f-9929-42dc3ad33589 admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:20:02.698762 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb7f48c-91df-432f-9929-42dc3ad33589 admin admin] PATH_INFO: `/v3/users` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:20:02.700934 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb7f48c-91df-432f-9929-42dc3ad33589 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:20:02.702950 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb7f48c-91df-432f-9929-42dc3ad33589 admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:20:02.752042 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6996472b-a42a-4d1f-b3d8-e85047ed86ac service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:20:02.753442 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6996472b-a42a-4d1f-b3d8-e85047ed86ac service cinder] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:20:02.766077 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1091/2164] 10.222.0.23 () {60 vars in 1365 bytes} [Thu May 19 12:20:02 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 287 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 12:20:02.841792 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1074/2165] 10.222.0.23 () {60 vars in 1143 bytes} [Thu May 19 12:20:02 2022] POST /identity/v3/users => generated 292 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 12:20:02.846130 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6996472b-a42a-4d1f-b3d8-e85047ed86ac service cinder] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:20:02.864265 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6996472b-a42a-4d1f-b3d8-e85047ed86ac service cinder] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:20:02.871061 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738620ed-1a7b-45e9-95ab-4c58bf8c441e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:20:02.871541 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738620ed-1a7b-45e9-95ab-4c58bf8c441e admin admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:20:02.872060 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738620ed-1a7b-45e9-95ab-4c58bf8c441e admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:20:02.872350 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738620ed-1a7b-45e9-95ab-4c58bf8c441e admin admin] PATH_INFO: `/v3/roles` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:20:02.873107 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-738620ed-1a7b-45e9-95ab-4c58bf8c441e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:20:02.874951 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-738620ed-1a7b-45e9-95ab-4c58bf8c441e admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:20:02.890024 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb7f48c-91df-432f-9929-42dc3ad33589 admin admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:20:02.897525 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1092/2166] 10.222.0.23 () {58 vars in 1121 bytes} [Thu May 19 12:20:02 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 12:20:02.900545 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738620ed-1a7b-45e9-95ab-4c58bf8c441e admin admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:20:02.903540 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb7f48c-91df-432f-9929-42dc3ad33589 admin admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:20:02.907870 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6757b448-1d4b-4b23-b0aa-714262ecc4e9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8800254c0e784e42a791fb9abc274d79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd70d25bf69314820b8aaecb65576978a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:20:02.909297 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6757b448-1d4b-4b23-b0aa-714262ecc4e9 service cinder] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:20:02.909660 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6757b448-1d4b-4b23-b0aa-714262ecc4e9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:20:02.910357 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6757b448-1d4b-4b23-b0aa-714262ecc4e9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:20:02.912499 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738620ed-1a7b-45e9-95ab-4c58bf8c441e admin admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:20:02.913316 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-6757b448-1d4b-4b23-b0aa-714262ecc4e9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:20:02.918999 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c33d276-d58b-45bb-8610-1f3a9f6724d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:20:02.919810 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c33d276-d58b-45bb-8610-1f3a9f6724d9 admin admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:20:02.920172 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c33d276-d58b-45bb-8610-1f3a9f6724d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:20:02.931870 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c33d276-d58b-45bb-8610-1f3a9f6724d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:20:02.933959 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c33d276-d58b-45bb-8610-1f3a9f6724d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:20:02.938029 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c33d276-d58b-45bb-8610-1f3a9f6724d9 admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:20:02.965186 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1093/2167] 10.222.0.23 () {58 vars in 1121 bytes} [Thu May 19 12:20:02 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 19 12:20:02.970087 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c33d276-d58b-45bb-8610-1f3a9f6724d9 admin admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:20:02.984650 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c33d276-d58b-45bb-8610-1f3a9f6724d9 admin admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:20:02.994133 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29eaa020-1587-403f-803e-5ed1887fa62c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:20:02.995359 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29eaa020-1587-403f-803e-5ed1887fa62c admin admin] REQUEST_METHOD: `PUT` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:20:02.996209 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29eaa020-1587-403f-803e-5ed1887fa62c admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:20:02.996671 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29eaa020-1587-403f-803e-5ed1887fa62c admin admin] PATH_INFO: `/v3/projects/a84ec302953a407ab2b13e19f68a0192/users/a59f390d41944cf7a7d7afdf29b0abae/roles/259eb400cdb4439db72d14456ae9cd08` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:20:03.006307 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6757b448-1d4b-4b23-b0aa-714262ecc4e9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:20:03.009170 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6757b448-1d4b-4b23-b0aa-714262ecc4e9 service cinder] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:20:03.023274 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29eaa020-1587-403f-803e-5ed1887fa62c admin admin] RBAC: Authorizing `identity:create_grant(project_id=a84ec302953a407ab2b13e19f68a0192, user_id=a59f390d41944cf7a7d7afdf29b0abae, role_id=259eb400cdb4439db72d14456ae9cd08)` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:20:03.029738 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1075/2168] 10.222.0.23 () {60 vars in 1365 bytes} [Thu May 19 12:20:02 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 12:20:03.031139 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29eaa020-1587-403f-803e-5ed1887fa62c admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:20:03.118316 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1094/2169] 10.222.0.23 () {60 vars in 1482 bytes} [Thu May 19 12:20:02 2022] PUT /identity/v3/projects/a84ec302953a407ab2b13e19f68a0192/users/a59f390d41944cf7a7d7afdf29b0abae/roles/259eb400cdb4439db72d14456ae9cd08 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:20:03.128377 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a930a02-373c-4213-b2f2-e81f9d6e808a None None] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:20:03.128727 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a930a02-373c-4213-b2f2-e81f9d6e808a None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:20:03.129021 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a930a02-373c-4213-b2f2-e81f9d6e808a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:20:03.164686 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.auth.core [None req-8a930a02-373c-4213-b2f2-e81f9d6e808a None None] MFA Rules not processed for user `a59f390d41944cf7a7d7afdf29b0abae`. Rule list: `[]` (Enabled: `True`). {{(pid=80539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:20:03.166472 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-8a930a02-373c-4213-b2f2-e81f9d6e808a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:20:03.173503 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:20:03.267532 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1076/2170] 10.222.0.23 () {58 vars in 957 bytes} [Thu May 19 12:20:03 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:20:03.281610 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29eaa020-1587-403f-803e-5ed1887fa62c admin admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:20:03.283852 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-29eaa020-1587-403f-803e-5ed1887fa62c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:20:03.400942 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29eaa020-1587-403f-803e-5ed1887fa62c admin admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:20:03.405165 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf90569-e4da-46df-acb1-c3d7e33fa312 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8800254c0e784e42a791fb9abc274d79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd70d25bf69314820b8aaecb65576978a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:20:03.405838 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf90569-e4da-46df-acb1-c3d7e33fa312 service cinder] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:20:03.406183 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf90569-e4da-46df-acb1-c3d7e33fa312 service cinder] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:20:03.406473 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf90569-e4da-46df-acb1-c3d7e33fa312 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:20:03.421156 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf90569-e4da-46df-acb1-c3d7e33fa312 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:20:03.422845 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf90569-e4da-46df-acb1-c3d7e33fa312 service cinder] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:20:03.436717 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1095/2171] 10.222.0.23 () {60 vars in 1365 bytes} [Thu May 19 12:20:03 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:20:04.627389 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:20:05.297381 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:20:06.597211 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:20:07.104406 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c36966-1deb-45cd-adc3-11f7767a67e6 None None] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:20:07.105149 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c36966-1deb-45cd-adc3-11f7767a67e6 None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:20:07.105359 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c36966-1deb-45cd-adc3-11f7767a67e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:20:07.137702 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.auth.core [None req-57c36966-1deb-45cd-adc3-11f7767a67e6 None None] MFA Rules not processed for user `3cdc4cc4d83b4495aa44de70147a001c`. Rule list: `[]` (Enabled: `True`). {{(pid=80539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:20:07.138923 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-57c36966-1deb-45cd-adc3-11f7767a67e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:20:07.212363 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1077/2172] 10.222.0.23 () {58 vars in 957 bytes} [Thu May 19 12:20:07 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:20:07.228075 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf90569-e4da-46df-acb1-c3d7e33fa312 service cinder] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:20:07.229572 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-4bf90569-e4da-46df-acb1-c3d7e33fa312 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:20:07.397847 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf90569-e4da-46df-acb1-c3d7e33fa312 service cinder] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:20:07.406166 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea42f67-f3d6-4896-b096-e4c9d5807f2b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34a98d75d84b4c67981377e0f26eca62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd70d25bf69314820b8aaecb65576978a', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:20:07.406937 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea42f67-f3d6-4896-b096-e4c9d5807f2b service nova] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:20:07.407302 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea42f67-f3d6-4896-b096-e4c9d5807f2b service nova] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:20:07.407632 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea42f67-f3d6-4896-b096-e4c9d5807f2b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:20:07.418579 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea42f67-f3d6-4896-b096-e4c9d5807f2b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:20:07.421348 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea42f67-f3d6-4896-b096-e4c9d5807f2b service nova] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:20:07.426878 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:20:07.435341 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1096/2173] 10.222.0.23 () {60 vars in 1365 bytes} [Thu May 19 12:20:07 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 209 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:20:08.400203 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:20:08.697105 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:20:08.869750 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:20:09.719851 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:20:10.222754 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:20:10.293211 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40423f18-048a-4a09-be69-d02df67c967b None None] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:20:10.293482 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40423f18-048a-4a09-be69-d02df67c967b None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:20:10.293763 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40423f18-048a-4a09-be69-d02df67c967b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:20:10.313350 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.auth.core [None req-40423f18-048a-4a09-be69-d02df67c967b None None] MFA Rules not processed for user `3cdc4cc4d83b4495aa44de70147a001c`. Rule list: `[]` (Enabled: `True`). {{(pid=80539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:20:10.314633 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-40423f18-048a-4a09-be69-d02df67c967b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:20:10.322803 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1078/2174] 10.222.0.23 () {58 vars in 957 bytes} [Thu May 19 12:20:10 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:20:10.599889 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:20:10.794185 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:20:10.846288 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:20:11.092096 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:20:13.402452 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e1261b-fb09-4fd4-aeb0-36dac0d8d205 None None] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:20:13.403028 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e1261b-fb09-4fd4-aeb0-36dac0d8d205 None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:20:13.403028 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e1261b-fb09-4fd4-aeb0-36dac0d8d205 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:20:13.421663 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.auth.core [None req-d4e1261b-fb09-4fd4-aeb0-36dac0d8d205 None None] MFA Rules not processed for user `858ef1572f064c248592e92625b228f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:20:13.423953 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-d4e1261b-fb09-4fd4-aeb0-36dac0d8d205 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:20:13.467845 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40423f18-048a-4a09-be69-d02df67c967b None None] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:20:13.485745 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1097/2175] 10.222.0.23 () {58 vars in 957 bytes} [Thu May 19 12:20:13 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:20:13.537744 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40423f18-048a-4a09-be69-d02df67c967b None None] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:20:13.543473 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b2f508-b4eb-4694-9471-679dec55b414 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:20:13.544391 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b2f508-b4eb-4694-9471-679dec55b414 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:20:13.544678 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b2f508-b4eb-4694-9471-679dec55b414 admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:20:13.544965 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b2f508-b4eb-4694-9471-679dec55b414 admin admin] PATH_INFO: `/v3/users/45c84d2b727f47e380c6a78423f170ff` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:20:13.546203 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b2f508-b4eb-4694-9471-679dec55b414 admin admin] RBAC: Authorizing `identity:delete_user(user_id=45c84d2b727f47e380c6a78423f170ff)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:20:13.548264 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b2f508-b4eb-4694-9471-679dec55b414 admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:20:13.557187 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:20:13.821347 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-38b2f508-b4eb-4694-9471-679dec55b414 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '45c84d2b727f47e380c6a78423f170ff'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:20:13.843531 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-38b2f508-b4eb-4694-9471-679dec55b414 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '45c84d2b727f47e380c6a78423f170ff'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:20:13.844375 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-38b2f508-b4eb-4694-9471-679dec55b414 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '45c84d2b727f47e380c6a78423f170ff'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:20:14.040308 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-38b2f508-b4eb-4694-9471-679dec55b414 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '45c84d2b727f47e380c6a78423f170ff'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:20:14.052634 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-38b2f508-b4eb-4694-9471-679dec55b414 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '45c84d2b727f47e380c6a78423f170ff'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:20:14.061616 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1079/2176] 10.222.0.23 () {58 vars in 1223 bytes} [Thu May 19 12:20:13 2022] DELETE /identity/v3/users/45c84d2b727f47e380c6a78423f170ff => generated 0 bytes in 594 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 12:20:14.072736 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11d24d8-35e3-440d-9131-6d3b5bc40524 None None] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:20:14.072736 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11d24d8-35e3-440d-9131-6d3b5bc40524 None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:20:14.072736 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11d24d8-35e3-440d-9131-6d3b5bc40524 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:20:14.089863 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.auth.core [None req-c11d24d8-35e3-440d-9131-6d3b5bc40524 None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:20:14.091113 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-c11d24d8-35e3-440d-9131-6d3b5bc40524 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:20:14.154718 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1098/2177] 10.222.0.23 () {58 vars in 957 bytes} [Thu May 19 12:20:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:20:14.164866 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b2f508-b4eb-4694-9471-679dec55b414 admin admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:20:14.166493 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-38b2f508-b4eb-4694-9471-679dec55b414 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:20:14.224946 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b2f508-b4eb-4694-9471-679dec55b414 admin admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:20:14.229077 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6192215-9dbe-4bba-b25a-680308f85006 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a00b3fa7497456895e5e0bf105c4fc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd70d25bf69314820b8aaecb65576978a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:20:14.229752 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6192215-9dbe-4bba-b25a-680308f85006 service neutron] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:20:14.230377 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6192215-9dbe-4bba-b25a-680308f85006 service neutron] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:20:14.230565 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6192215-9dbe-4bba-b25a-680308f85006 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:20:14.240177 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6192215-9dbe-4bba-b25a-680308f85006 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:20:14.241673 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6192215-9dbe-4bba-b25a-680308f85006 service neutron] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:20:14.252494 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1080/2178] 10.222.0.23 () {60 vars in 1365 bytes} [Thu May 19 12:20:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 12:20:15.049977 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11d24d8-35e3-440d-9131-6d3b5bc40524 None None] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:20:15.051653 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-c11d24d8-35e3-440d-9131-6d3b5bc40524 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:20:15.066022 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11d24d8-35e3-440d-9131-6d3b5bc40524 None None] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:20:15.072769 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e988f1-b359-430c-bb00-a03857c2e36c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:20:15.073889 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e988f1-b359-430c-bb00-a03857c2e36c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:20:15.074175 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e988f1-b359-430c-bb00-a03857c2e36c admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:20:15.074452 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e988f1-b359-430c-bb00-a03857c2e36c admin admin] PATH_INFO: `/v3/projects/ca857a614c1745df8b8a1f91a46a6394` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:20:15.075708 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e988f1-b359-430c-bb00-a03857c2e36c admin admin] RBAC: Authorizing `identity:delete_project(project_id=ca857a614c1745df8b8a1f91a46a6394)` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:20:15.077719 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e988f1-b359-430c-bb00-a03857c2e36c admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:20:15.260176 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:20:15.277625 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-30e988f1-b359-430c-bb00-a03857c2e36c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ca857a614c1745df8b8a1f91a46a6394', 'request_id': 'req-30e988f1-b359-430c-bb00-a03857c2e36c'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:20:15.283851 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-30e988f1-b359-430c-bb00-a03857c2e36c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ca857a614c1745df8b8a1f91a46a6394', 'request_id': 'req-30e988f1-b359-430c-bb00-a03857c2e36c'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:20:15.290158 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-30e988f1-b359-430c-bb00-a03857c2e36c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ca857a614c1745df8b8a1f91a46a6394', 'request_id': 'req-30e988f1-b359-430c-bb00-a03857c2e36c'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:20:15.463256 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-30e988f1-b359-430c-bb00-a03857c2e36c admin admin] The token cache is being invalidate because project ca857a614c1745df8b8a1f91a46a6394 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80540) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 12:20:15.463713 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-30e988f1-b359-430c-bb00-a03857c2e36c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:20:15.464406 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-30e988f1-b359-430c-bb00-a03857c2e36c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:20:15.465963 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1099/2179] 10.222.0.23 () {58 vars in 1232 bytes} [Thu May 19 12:20:15 2022] DELETE /identity/v3/projects/ca857a614c1745df8b8a1f91a46a6394 => generated 0 bytes in 417 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:20:15.469904 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6192215-9dbe-4bba-b25a-680308f85006 service neutron] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:20:15.480587 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6192215-9dbe-4bba-b25a-680308f85006 service neutron] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:20:15.540105 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d10614-0cbb-44d2-b660-afc0e5069a36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:20:15.541095 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d10614-0cbb-44d2-b660-afc0e5069a36 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:20:15.541314 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d10614-0cbb-44d2-b660-afc0e5069a36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:20:15.541592 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d10614-0cbb-44d2-b660-afc0e5069a36 admin admin] PATH_INFO: `/v3/users/281e1bc4c0fc478ba98ee61f5b731c5a` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:20:15.542865 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d10614-0cbb-44d2-b660-afc0e5069a36 admin admin] RBAC: Authorizing `identity:delete_user(user_id=281e1bc4c0fc478ba98ee61f5b731c5a)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:20:15.544860 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d10614-0cbb-44d2-b660-afc0e5069a36 admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:20:15.783547 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:20:15.798623 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-43d10614-0cbb-44d2-b660-afc0e5069a36 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '281e1bc4c0fc478ba98ee61f5b731c5a'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:20:15.812031 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-43d10614-0cbb-44d2-b660-afc0e5069a36 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '281e1bc4c0fc478ba98ee61f5b731c5a'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:20:15.812745 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-43d10614-0cbb-44d2-b660-afc0e5069a36 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '281e1bc4c0fc478ba98ee61f5b731c5a'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:20:15.919905 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-43d10614-0cbb-44d2-b660-afc0e5069a36 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '281e1bc4c0fc478ba98ee61f5b731c5a'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:20:15.927633 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-43d10614-0cbb-44d2-b660-afc0e5069a36 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '281e1bc4c0fc478ba98ee61f5b731c5a'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:20:15.934114 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1081/2180] 10.222.0.23 () {58 vars in 1223 bytes} [Thu May 19 12:20:15 2022] DELETE /identity/v3/users/281e1bc4c0fc478ba98ee61f5b731c5a => generated 0 bytes in 465 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:20:16.512142 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e988f1-b359-430c-bb00-a03857c2e36c admin admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:20:16.513662 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-30e988f1-b359-430c-bb00-a03857c2e36c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:20:16.528248 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e988f1-b359-430c-bb00-a03857c2e36c admin admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:20:16.602039 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22e74c9-e5ac-49e4-9afb-dd3c4c742bc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:20:16.603122 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22e74c9-e5ac-49e4-9afb-dd3c4c742bc6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:20:16.603543 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22e74c9-e5ac-49e4-9afb-dd3c4c742bc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:20:16.603962 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22e74c9-e5ac-49e4-9afb-dd3c4c742bc6 admin admin] PATH_INFO: `/v3/projects/3651442a6e6144fb99ae49c3db1107be` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:20:16.605700 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22e74c9-e5ac-49e4-9afb-dd3c4c742bc6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3651442a6e6144fb99ae49c3db1107be)` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:20:16.607896 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22e74c9-e5ac-49e4-9afb-dd3c4c742bc6 admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:20:16.812174 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-d22e74c9-e5ac-49e4-9afb-dd3c4c742bc6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3651442a6e6144fb99ae49c3db1107be', 'request_id': 'req-d22e74c9-e5ac-49e4-9afb-dd3c4c742bc6'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:20:16.819296 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-d22e74c9-e5ac-49e4-9afb-dd3c4c742bc6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3651442a6e6144fb99ae49c3db1107be', 'request_id': 'req-d22e74c9-e5ac-49e4-9afb-dd3c4c742bc6'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:20:16.833060 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-d22e74c9-e5ac-49e4-9afb-dd3c4c742bc6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3651442a6e6144fb99ae49c3db1107be', 'request_id': 'req-d22e74c9-e5ac-49e4-9afb-dd3c4c742bc6'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:20:16.966698 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-d22e74c9-e5ac-49e4-9afb-dd3c4c742bc6 admin admin] The token cache is being invalidate because project 3651442a6e6144fb99ae49c3db1107be was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80540) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 12:20:16.968362 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-d22e74c9-e5ac-49e4-9afb-dd3c4c742bc6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:20:16.968362 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-d22e74c9-e5ac-49e4-9afb-dd3c4c742bc6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:20:16.969314 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1100/2181] 10.222.0.23 () {58 vars in 1232 bytes} [Thu May 19 12:20:16 2022] DELETE /identity/v3/projects/3651442a6e6144fb99ae49c3db1107be => generated 0 bytes in 458 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:20:16.973504 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d10614-0cbb-44d2-b660-afc0e5069a36 admin admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:20:16.974975 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-43d10614-0cbb-44d2-b660-afc0e5069a36 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:20:16.988616 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d10614-0cbb-44d2-b660-afc0e5069a36 admin admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:20:17.047344 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fc8577-9ac3-4148-a313-19e5689eab41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:20:17.048260 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fc8577-9ac3-4148-a313-19e5689eab41 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:20:17.048545 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fc8577-9ac3-4148-a313-19e5689eab41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:20:17.048820 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fc8577-9ac3-4148-a313-19e5689eab41 admin admin] PATH_INFO: `/v3/users/0d7e8c2c7ef448edbf173fe8ea50da77` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:20:17.050051 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34fc8577-9ac3-4148-a313-19e5689eab41 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0d7e8c2c7ef448edbf173fe8ea50da77)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:20:17.052005 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34fc8577-9ac3-4148-a313-19e5689eab41 admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:20:17.461794 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-34fc8577-9ac3-4148-a313-19e5689eab41 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0d7e8c2c7ef448edbf173fe8ea50da77'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:20:17.474335 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-34fc8577-9ac3-4148-a313-19e5689eab41 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0d7e8c2c7ef448edbf173fe8ea50da77'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:20:17.475067 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-34fc8577-9ac3-4148-a313-19e5689eab41 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0d7e8c2c7ef448edbf173fe8ea50da77'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:20:17.606096 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-34fc8577-9ac3-4148-a313-19e5689eab41 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0d7e8c2c7ef448edbf173fe8ea50da77'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:20:17.614295 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-34fc8577-9ac3-4148-a313-19e5689eab41 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0d7e8c2c7ef448edbf173fe8ea50da77'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:20:17.621723 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1082/2182] 10.222.0.23 () {58 vars in 1223 bytes} [Thu May 19 12:20:16 2022] DELETE /identity/v3/users/0d7e8c2c7ef448edbf173fe8ea50da77 => generated 0 bytes in 649 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:20:17.860186 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22e74c9-e5ac-49e4-9afb-dd3c4c742bc6 admin admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:20:17.862201 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-d22e74c9-e5ac-49e4-9afb-dd3c4c742bc6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:20:17.878862 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22e74c9-e5ac-49e4-9afb-dd3c4c742bc6 admin admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:20:17.935034 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:20:17.958020 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc32f2e-5f2b-4954-8421-abb12f5f4063 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:20:17.958160 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc32f2e-5f2b-4954-8421-abb12f5f4063 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:20:17.958391 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc32f2e-5f2b-4954-8421-abb12f5f4063 admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:20:17.959025 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc32f2e-5f2b-4954-8421-abb12f5f4063 admin admin] PATH_INFO: `/v3/users/a59f390d41944cf7a7d7afdf29b0abae` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:20:17.960633 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc32f2e-5f2b-4954-8421-abb12f5f4063 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a59f390d41944cf7a7d7afdf29b0abae)` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:20:17.963483 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc32f2e-5f2b-4954-8421-abb12f5f4063 admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:20:18.398462 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-6cc32f2e-5f2b-4954-8421-abb12f5f4063 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a59f390d41944cf7a7d7afdf29b0abae'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:20:18.411618 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-6cc32f2e-5f2b-4954-8421-abb12f5f4063 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a59f390d41944cf7a7d7afdf29b0abae'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:20:18.412433 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-6cc32f2e-5f2b-4954-8421-abb12f5f4063 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a59f390d41944cf7a7d7afdf29b0abae'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:20:18.496561 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fc8577-9ac3-4148-a313-19e5689eab41 admin admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:20:18.498264 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-34fc8577-9ac3-4148-a313-19e5689eab41 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:20:18.514976 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fc8577-9ac3-4148-a313-19e5689eab41 admin admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:20:18.520233 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4987947f-4439-4879-8dcf-111956c9a0c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:20:18.521249 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4987947f-4439-4879-8dcf-111956c9a0c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:20:18.521800 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4987947f-4439-4879-8dcf-111956c9a0c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:20:18.522169 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4987947f-4439-4879-8dcf-111956c9a0c2 admin admin] PATH_INFO: `/v3/projects/60cd52fe4a574a418d586e6a38a38295` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:20:18.524404 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4987947f-4439-4879-8dcf-111956c9a0c2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=60cd52fe4a574a418d586e6a38a38295)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:20:18.526379 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4987947f-4439-4879-8dcf-111956c9a0c2 admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:20:18.560315 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:20:18.639367 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-6cc32f2e-5f2b-4954-8421-abb12f5f4063 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a59f390d41944cf7a7d7afdf29b0abae'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:20:18.655744 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-6cc32f2e-5f2b-4954-8421-abb12f5f4063 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a59f390d41944cf7a7d7afdf29b0abae'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:20:18.666095 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1101/2183] 10.222.0.23 () {58 vars in 1223 bytes} [Thu May 19 12:20:17 2022] DELETE /identity/v3/users/a59f390d41944cf7a7d7afdf29b0abae => generated 0 bytes in 806 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:20:18.670629 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6330df86-a788-43c3-ba68-bd3d407f4c13 None None] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:20:18.671038 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6330df86-a788-43c3-ba68-bd3d407f4c13 None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:20:18.671395 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6330df86-a788-43c3-ba68-bd3d407f4c13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:20:18.690688 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.auth.core [None req-6330df86-a788-43c3-ba68-bd3d407f4c13 None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:20:18.692143 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-6330df86-a788-43c3-ba68-bd3d407f4c13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:20:18.725292 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-4987947f-4439-4879-8dcf-111956c9a0c2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '60cd52fe4a574a418d586e6a38a38295', 'request_id': 'req-4987947f-4439-4879-8dcf-111956c9a0c2'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:20:18.734607 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-4987947f-4439-4879-8dcf-111956c9a0c2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '60cd52fe4a574a418d586e6a38a38295', 'request_id': 'req-4987947f-4439-4879-8dcf-111956c9a0c2'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:20:18.766042 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1102/2184] 10.222.0.23 () {58 vars in 957 bytes} [Thu May 19 12:20:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:20:18.774686 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6330df86-a788-43c3-ba68-bd3d407f4c13 None None] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:20:18.777120 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-6330df86-a788-43c3-ba68-bd3d407f4c13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:20:18.840273 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6330df86-a788-43c3-ba68-bd3d407f4c13 None None] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:20:18.893998 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0caae3-e86c-44e0-92cc-a5da94ed6849 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a00b3fa7497456895e5e0bf105c4fc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd70d25bf69314820b8aaecb65576978a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:20:18.894621 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0caae3-e86c-44e0-92cc-a5da94ed6849 service neutron] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:20:18.894869 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0caae3-e86c-44e0-92cc-a5da94ed6849 service neutron] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:20:18.895285 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0caae3-e86c-44e0-92cc-a5da94ed6849 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:20:18.898493 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-4987947f-4439-4879-8dcf-111956c9a0c2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '60cd52fe4a574a418d586e6a38a38295', 'request_id': 'req-4987947f-4439-4879-8dcf-111956c9a0c2'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:20:18.915897 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-4987947f-4439-4879-8dcf-111956c9a0c2 admin admin] The token cache is being invalidate because project 60cd52fe4a574a418d586e6a38a38295 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80539) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 12:20:18.916190 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-4987947f-4439-4879-8dcf-111956c9a0c2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:20:18.916871 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-4987947f-4439-4879-8dcf-111956c9a0c2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:20:18.918451 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1083/2185] 10.222.0.23 () {58 vars in 1232 bytes} [Thu May 19 12:20:18 2022] DELETE /identity/v3/projects/60cd52fe4a574a418d586e6a38a38295 => generated 0 bytes in 423 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:20:18.924535 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3bc981-6d21-4fd5-897e-6e2aa2e5bd4a None None] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:20:18.924826 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3bc981-6d21-4fd5-897e-6e2aa2e5bd4a None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:20:18.925153 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3bc981-6d21-4fd5-897e-6e2aa2e5bd4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:20:18.947482 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.auth.core [None req-db3bc981-6d21-4fd5-897e-6e2aa2e5bd4a None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:20:18.947482 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-db3bc981-6d21-4fd5-897e-6e2aa2e5bd4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:20:18.971256 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0caae3-e86c-44e0-92cc-a5da94ed6849 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:20:18.973063 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0caae3-e86c-44e0-92cc-a5da94ed6849 service neutron] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:20:18.988494 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1103/2186] 10.222.0.23 () {60 vars in 1365 bytes} [Thu May 19 12:20:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 214 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:20:19.038118 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1084/2187] 10.222.0.23 () {58 vars in 957 bytes} [Thu May 19 12:20:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:20:19.048169 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5018f47-fef3-452d-af9f-7273ec48ae55 None None] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:20:19.048342 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5018f47-fef3-452d-af9f-7273ec48ae55 None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:20:19.048771 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5018f47-fef3-452d-af9f-7273ec48ae55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:20:19.079980 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.auth.core [None req-f5018f47-fef3-452d-af9f-7273ec48ae55 None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:20:19.080609 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-f5018f47-fef3-452d-af9f-7273ec48ae55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:20:19.096336 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3bc981-6d21-4fd5-897e-6e2aa2e5bd4a None None] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:20:19.103190 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1104/2188] 10.222.0.23 () {58 vars in 957 bytes} [Thu May 19 12:20:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:20:19.110837 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3bc981-6d21-4fd5-897e-6e2aa2e5bd4a None None] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:20:19.117277 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e0a824-c8f5-4ba6-8f03-52aaba802667 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:20:19.117973 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e0a824-c8f5-4ba6-8f03-52aaba802667 admin admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:20:19.118340 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e0a824-c8f5-4ba6-8f03-52aaba802667 admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:20:19.118681 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e0a824-c8f5-4ba6-8f03-52aaba802667 admin admin] PATH_INFO: `/v3/domains` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:20:19.120001 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e0a824-c8f5-4ba6-8f03-52aaba802667 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:20:19.120388 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e0a824-c8f5-4ba6-8f03-52aaba802667 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:20:19.122310 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e0a824-c8f5-4ba6-8f03-52aaba802667 admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:20:19.154761 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1085/2189] 10.222.0.23 () {58 vars in 1152 bytes} [Thu May 19 12:20:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:20:19.156731 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5018f47-fef3-452d-af9f-7273ec48ae55 None None] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:20:19.210212 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5018f47-fef3-452d-af9f-7273ec48ae55 None None] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:20:19.249692 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bcce17d-7a9d-4c14-8ec3-165afaa98ab9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:20:19.251277 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcce17d-7a9d-4c14-8ec3-165afaa98ab9 admin admin] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:20:19.252746 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcce17d-7a9d-4c14-8ec3-165afaa98ab9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:20:19.253455 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcce17d-7a9d-4c14-8ec3-165afaa98ab9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:20:19.254965 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bcce17d-7a9d-4c14-8ec3-165afaa98ab9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:20:19.261628 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bcce17d-7a9d-4c14-8ec3-165afaa98ab9 admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:20:19.504103 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e0a824-c8f5-4ba6-8f03-52aaba802667 admin admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:20:19.505856 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1105/2190] 10.222.0.23 () {60 vars in 1152 bytes} [Thu May 19 12:20:19 2022] POST /identity/v3/projects => generated 373 bytes in 351 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 12:20:19.518463 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e0a824-c8f5-4ba6-8f03-52aaba802667 admin admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:20:19.583734 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:20:19.609956 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299e287d-b3d0-4b12-8e28-b02ac6ab2404 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:20:19.613292 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299e287d-b3d0-4b12-8e28-b02ac6ab2404 admin admin] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:20:19.613292 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299e287d-b3d0-4b12-8e28-b02ac6ab2404 admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:20:19.613292 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299e287d-b3d0-4b12-8e28-b02ac6ab2404 admin admin] PATH_INFO: `/v3/users` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:20:19.614044 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299e287d-b3d0-4b12-8e28-b02ac6ab2404 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:20:19.616203 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299e287d-b3d0-4b12-8e28-b02ac6ab2404 admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:20:19.818095 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1086/2191] 10.222.0.23 () {60 vars in 1143 bytes} [Thu May 19 12:20:19 2022] POST /identity/v3/users => generated 287 bytes in 315 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 12:20:19.823570 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bcce17d-7a9d-4c14-8ec3-165afaa98ab9 admin admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:20:19.833897 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bcce17d-7a9d-4c14-8ec3-165afaa98ab9 admin admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:20:19.839178 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b56661-1619-4269-b491-19b3429ef134 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:20:19.839899 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b56661-1619-4269-b491-19b3429ef134 admin admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:20:19.840192 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b56661-1619-4269-b491-19b3429ef134 admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:20:19.840504 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b56661-1619-4269-b491-19b3429ef134 admin admin] PATH_INFO: `/v3/roles` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:20:19.841282 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b56661-1619-4269-b491-19b3429ef134 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:20:19.842940 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b56661-1619-4269-b491-19b3429ef134 admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:20:19.855772 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1106/2192] 10.222.0.23 () {58 vars in 1121 bytes} [Thu May 19 12:20:19 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 12:20:19.859516 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299e287d-b3d0-4b12-8e28-b02ac6ab2404 admin admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:20:19.870904 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299e287d-b3d0-4b12-8e28-b02ac6ab2404 admin admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:20:19.876538 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e05003-0753-462e-8110-84ad8e7d7df0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:20:19.877259 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e05003-0753-462e-8110-84ad8e7d7df0 admin admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:20:19.877575 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e05003-0753-462e-8110-84ad8e7d7df0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:20:19.877854 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e05003-0753-462e-8110-84ad8e7d7df0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:20:19.878620 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e05003-0753-462e-8110-84ad8e7d7df0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:20:19.880295 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e05003-0753-462e-8110-84ad8e7d7df0 admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:20:19.892983 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1087/2193] 10.222.0.23 () {58 vars in 1121 bytes} [Thu May 19 12:20:19 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 12:20:19.896860 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b56661-1619-4269-b491-19b3429ef134 admin admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:20:19.915955 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b56661-1619-4269-b491-19b3429ef134 admin admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:20:19.919798 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e05003-0753-462e-8110-84ad8e7d7df0 admin admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:20:19.920994 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147f5964-1265-48e5-a721-735ef56ff219 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:20:19.922333 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-56e05003-0753-462e-8110-84ad8e7d7df0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:20:19.922909 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147f5964-1265-48e5-a721-735ef56ff219 admin admin] REQUEST_METHOD: `PUT` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:20:19.923388 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147f5964-1265-48e5-a721-735ef56ff219 admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:20:19.923706 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147f5964-1265-48e5-a721-735ef56ff219 admin admin] PATH_INFO: `/v3/projects/39be464eb5ec429aa888dd81bbe31496/users/5db46f6216f347de9816b2227cbecae7/roles/259eb400cdb4439db72d14456ae9cd08` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:20:19.941371 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e05003-0753-462e-8110-84ad8e7d7df0 admin admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:20:19.948605 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19215383-3dda-4c84-8327-187854b1a790 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:20:19.949492 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19215383-3dda-4c84-8327-187854b1a790 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:20:19.949779 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19215383-3dda-4c84-8327-187854b1a790 admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:20:19.950055 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19215383-3dda-4c84-8327-187854b1a790 admin admin] PATH_INFO: `/v3/projects/a84ec302953a407ab2b13e19f68a0192` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:20:19.954049 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19215383-3dda-4c84-8327-187854b1a790 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a84ec302953a407ab2b13e19f68a0192)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:20:19.956372 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19215383-3dda-4c84-8327-187854b1a790 admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:20:19.983276 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147f5964-1265-48e5-a721-735ef56ff219 admin admin] RBAC: Authorizing `identity:create_grant(project_id=39be464eb5ec429aa888dd81bbe31496, user_id=5db46f6216f347de9816b2227cbecae7, role_id=259eb400cdb4439db72d14456ae9cd08)` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:20:19.986961 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147f5964-1265-48e5-a721-735ef56ff219 admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:20:20.160542 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1107/2194] 10.222.0.23 () {60 vars in 1482 bytes} [Thu May 19 12:20:19 2022] PUT /identity/v3/projects/39be464eb5ec429aa888dd81bbe31496/users/5db46f6216f347de9816b2227cbecae7/roles/259eb400cdb4439db72d14456ae9cd08 => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:20:20.162184 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef32c415-0b41-42a5-8e88-81b9c9724be9 None None] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:20:20.162487 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef32c415-0b41-42a5-8e88-81b9c9724be9 None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:20:20.162859 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef32c415-0b41-42a5-8e88-81b9c9724be9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:20:20.170341 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-19215383-3dda-4c84-8327-187854b1a790 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a84ec302953a407ab2b13e19f68a0192', 'request_id': 'req-19215383-3dda-4c84-8327-187854b1a790'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:20:20.181641 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-19215383-3dda-4c84-8327-187854b1a790 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a84ec302953a407ab2b13e19f68a0192', 'request_id': 'req-19215383-3dda-4c84-8327-187854b1a790'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:20:20.182999 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.auth.core [None req-ef32c415-0b41-42a5-8e88-81b9c9724be9 None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:20:20.184349 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-ef32c415-0b41-42a5-8e88-81b9c9724be9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:20:20.269489 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1108/2195] 10.222.0.23 () {58 vars in 957 bytes} [Thu May 19 12:20:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:20:20.278925 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef32c415-0b41-42a5-8e88-81b9c9724be9 None None] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:20:20.280615 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-ef32c415-0b41-42a5-8e88-81b9c9724be9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:20:20.337928 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-19215383-3dda-4c84-8327-187854b1a790 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a84ec302953a407ab2b13e19f68a0192', 'request_id': 'req-19215383-3dda-4c84-8327-187854b1a790'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:20:20.347191 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef32c415-0b41-42a5-8e88-81b9c9724be9 None None] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:20:20.350275 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-19215383-3dda-4c84-8327-187854b1a790 admin admin] The token cache is being invalidate because project a84ec302953a407ab2b13e19f68a0192 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80539) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 12:20:20.350664 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-19215383-3dda-4c84-8327-187854b1a790 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:20:20.351964 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-19215383-3dda-4c84-8327-187854b1a790 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:20:20.352294 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9bc7ee-3210-43ed-82f3-f3ccf03a6b90 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a00b3fa7497456895e5e0bf105c4fc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd70d25bf69314820b8aaecb65576978a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:20:20.352903 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9bc7ee-3210-43ed-82f3-f3ccf03a6b90 service neutron] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:20:20.353189 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9bc7ee-3210-43ed-82f3-f3ccf03a6b90 service neutron] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:20:20.353470 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9bc7ee-3210-43ed-82f3-f3ccf03a6b90 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:20:20.354242 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1088/2196] 10.222.0.23 () {58 vars in 1232 bytes} [Thu May 19 12:20:19 2022] DELETE /identity/v3/projects/a84ec302953a407ab2b13e19f68a0192 => generated 0 bytes in 437 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:20:20.359499 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-9a9bc7ee-3210-43ed-82f3-f3ccf03a6b90 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:20:20.359696 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7811c0-3369-40c8-a004-388060053001 None None] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:20:20.360091 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7811c0-3369-40c8-a004-388060053001 None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:20:20.360466 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7811c0-3369-40c8-a004-388060053001 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:20:20.375873 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9bc7ee-3210-43ed-82f3-f3ccf03a6b90 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:20:20.376896 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9bc7ee-3210-43ed-82f3-f3ccf03a6b90 service neutron] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:20:20.381076 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.auth.core [None req-6c7811c0-3369-40c8-a004-388060053001 None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:20:20.382935 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-6c7811c0-3369-40c8-a004-388060053001 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:20:20.389201 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1109/2197] 10.222.0.23 () {60 vars in 1365 bytes} [Thu May 19 12:20:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 12:20:20.394469 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1089/2198] 10.222.0.23 () {58 vars in 957 bytes} [Thu May 19 12:20:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:20:20.403647 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0821d246-7000-4297-9312-b65609f955b4 None None] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:20:20.403982 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0821d246-7000-4297-9312-b65609f955b4 None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:20:20.404286 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0821d246-7000-4297-9312-b65609f955b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:20:20.424875 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.auth.core [None req-0821d246-7000-4297-9312-b65609f955b4 None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:20:20.426010 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-0821d246-7000-4297-9312-b65609f955b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:20:20.436264 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1110/2199] 10.222.0.23 () {58 vars in 957 bytes} [Thu May 19 12:20:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:20:20.440862 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7811c0-3369-40c8-a004-388060053001 None None] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:20:20.450962 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7811c0-3369-40c8-a004-388060053001 None None] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:20:20.455905 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8471c5-408f-4d94-b87f-299e2fc02255 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:20:20.456574 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8471c5-408f-4d94-b87f-299e2fc02255 admin admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:20:20.456860 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8471c5-408f-4d94-b87f-299e2fc02255 admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:20:20.457142 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8471c5-408f-4d94-b87f-299e2fc02255 admin admin] PATH_INFO: `/v3/domains` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:20:20.457881 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8471c5-408f-4d94-b87f-299e2fc02255 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:20:20.458187 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8471c5-408f-4d94-b87f-299e2fc02255 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:20:20.459817 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8471c5-408f-4d94-b87f-299e2fc02255 admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:20:20.475237 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1090/2200] 10.222.0.23 () {58 vars in 1152 bytes} [Thu May 19 12:20:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:20:20.477050 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0821d246-7000-4297-9312-b65609f955b4 None None] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:20:20.492840 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0821d246-7000-4297-9312-b65609f955b4 None None] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:20:20.493862 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4976e57a-0e90-42e2-b4c7-1f476952dd58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:20:20.494584 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4976e57a-0e90-42e2-b4c7-1f476952dd58 admin admin] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:20:20.494885 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4976e57a-0e90-42e2-b4c7-1f476952dd58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:20:20.495177 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4976e57a-0e90-42e2-b4c7-1f476952dd58 admin admin] PATH_INFO: `/v3/projects` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:20:20.497355 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4976e57a-0e90-42e2-b4c7-1f476952dd58 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:20:20.497908 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4976e57a-0e90-42e2-b4c7-1f476952dd58 admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:20:20.604937 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1111/2201] 10.222.0.23 () {60 vars in 1152 bytes} [Thu May 19 12:20:20 2022] POST /identity/v3/projects => generated 373 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 12:20:20.609968 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8471c5-408f-4d94-b87f-299e2fc02255 admin admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:20:20.635605 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8471c5-408f-4d94-b87f-299e2fc02255 admin admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:20:20.679292 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:DELETE=7,keystone:SELECT=89,keystone:INSERT=4 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:20:20.730114 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9490e649-6ae0-4e2b-8bf6-8cf5b2aec753 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:20:20.731096 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9490e649-6ae0-4e2b-8bf6-8cf5b2aec753 admin admin] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:20:20.731639 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9490e649-6ae0-4e2b-8bf6-8cf5b2aec753 admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:20:20.731985 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9490e649-6ae0-4e2b-8bf6-8cf5b2aec753 admin admin] PATH_INFO: `/v3/users` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:20:20.732738 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9490e649-6ae0-4e2b-8bf6-8cf5b2aec753 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:20:20.734738 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9490e649-6ae0-4e2b-8bf6-8cf5b2aec753 admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:20:20.897813 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1091/2202] 10.222.0.23 () {60 vars in 1143 bytes} [Thu May 19 12:20:20 2022] POST /identity/v3/users => generated 287 bytes in 288 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 12:20:20.902565 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4976e57a-0e90-42e2-b4c7-1f476952dd58 admin admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:20:20.913833 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4976e57a-0e90-42e2-b4c7-1f476952dd58 admin admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:20:20.919152 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf28e97-2225-4f15-93af-d428ed41ed2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:20:20.919875 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf28e97-2225-4f15-93af-d428ed41ed2c admin admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:20:20.920171 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf28e97-2225-4f15-93af-d428ed41ed2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:20:20.920459 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf28e97-2225-4f15-93af-d428ed41ed2c admin admin] PATH_INFO: `/v3/roles` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:20:20.921220 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf28e97-2225-4f15-93af-d428ed41ed2c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:20:20.922938 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf28e97-2225-4f15-93af-d428ed41ed2c admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:20:20.936477 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1112/2203] 10.222.0.23 () {58 vars in 1121 bytes} [Thu May 19 12:20:20 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 12:20:20.939909 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9490e649-6ae0-4e2b-8bf6-8cf5b2aec753 admin admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:20:20.961685 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9490e649-6ae0-4e2b-8bf6-8cf5b2aec753 admin admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:20:20.967076 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63715a91-444f-478e-9a55-ab67c772145e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:20:20.967775 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63715a91-444f-478e-9a55-ab67c772145e admin admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:20:20.968095 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63715a91-444f-478e-9a55-ab67c772145e admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:20:20.968498 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63715a91-444f-478e-9a55-ab67c772145e admin admin] PATH_INFO: `/v3/roles` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:20:20.969277 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63715a91-444f-478e-9a55-ab67c772145e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:20:20.970971 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63715a91-444f-478e-9a55-ab67c772145e admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:20:20.987639 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1092/2204] 10.222.0.23 () {58 vars in 1121 bytes} [Thu May 19 12:20:20 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 12:20:20.988947 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf28e97-2225-4f15-93af-d428ed41ed2c admin admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:20:21.001199 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf28e97-2225-4f15-93af-d428ed41ed2c admin admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:20:21.013083 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03931e92-7815-44c3-b264-4f4f2fd5f5cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:20:21.014182 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03931e92-7815-44c3-b264-4f4f2fd5f5cc admin admin] REQUEST_METHOD: `PUT` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:20:21.014498 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03931e92-7815-44c3-b264-4f4f2fd5f5cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:20:21.014923 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03931e92-7815-44c3-b264-4f4f2fd5f5cc admin admin] PATH_INFO: `/v3/projects/776e5ee8d7bf4f3c9cc37b6e30b5cd02/users/2064d45822fc43219fc7b2bc6a712f25/roles/259eb400cdb4439db72d14456ae9cd08` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:20:21.037700 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03931e92-7815-44c3-b264-4f4f2fd5f5cc admin admin] RBAC: Authorizing `identity:create_grant(project_id=776e5ee8d7bf4f3c9cc37b6e30b5cd02, user_id=2064d45822fc43219fc7b2bc6a712f25, role_id=259eb400cdb4439db72d14456ae9cd08)` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:20:21.040511 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03931e92-7815-44c3-b264-4f4f2fd5f5cc admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:20:21.554835 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1113/2205] 10.222.0.23 () {60 vars in 1482 bytes} [Thu May 19 12:20:20 2022] PUT /identity/v3/projects/776e5ee8d7bf4f3c9cc37b6e30b5cd02/users/2064d45822fc43219fc7b2bc6a712f25/roles/259eb400cdb4439db72d14456ae9cd08 => generated 0 bytes in 560 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:20:21.564080 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91125971-e2e4-4606-aec6-e8410e565c34 None None] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:20:21.564376 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91125971-e2e4-4606-aec6-e8410e565c34 None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:20:21.564662 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91125971-e2e4-4606-aec6-e8410e565c34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:20:21.593040 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.auth.core [None req-91125971-e2e4-4606-aec6-e8410e565c34 None None] MFA Rules not processed for user `2064d45822fc43219fc7b2bc6a712f25`. Rule list: `[]` (Enabled: `True`). {{(pid=80539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:20:21.594305 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-91125971-e2e4-4606-aec6-e8410e565c34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:20:21.672844 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1093/2206] 10.222.0.23 () {58 vars in 957 bytes} [Thu May 19 12:20:21 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:20:21.684568 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03931e92-7815-44c3-b264-4f4f2fd5f5cc admin admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:20:21.686535 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-03931e92-7815-44c3-b264-4f4f2fd5f5cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:20:21.753762 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03931e92-7815-44c3-b264-4f4f2fd5f5cc admin admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:20:21.757761 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53df6c4b-0585-4ec0-afc7-cb2b60309315 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8800254c0e784e42a791fb9abc274d79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd70d25bf69314820b8aaecb65576978a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:20:21.758434 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53df6c4b-0585-4ec0-afc7-cb2b60309315 service cinder] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:20:21.758730 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53df6c4b-0585-4ec0-afc7-cb2b60309315 service cinder] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:20:21.759013 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53df6c4b-0585-4ec0-afc7-cb2b60309315 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:20:21.769013 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53df6c4b-0585-4ec0-afc7-cb2b60309315 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:20:21.770510 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53df6c4b-0585-4ec0-afc7-cb2b60309315 service cinder] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:20:21.781025 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1114/2207] 10.222.0.23 () {60 vars in 1365 bytes} [Thu May 19 12:20:21 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:20:22.211883 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:20:23.081729 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a28b443-16c6-4920-9fd6-04c4ce30defd None None] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:20:23.082214 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a28b443-16c6-4920-9fd6-04c4ce30defd None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:20:23.082467 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a28b443-16c6-4920-9fd6-04c4ce30defd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:20:23.113106 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.auth.core [None req-4a28b443-16c6-4920-9fd6-04c4ce30defd None None] MFA Rules not processed for user `5db46f6216f347de9816b2227cbecae7`. Rule list: `[]` (Enabled: `True`). {{(pid=80539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:20:23.116362 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-4a28b443-16c6-4920-9fd6-04c4ce30defd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:20:23.207308 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1094/2208] 10.222.0.23 () {58 vars in 957 bytes} [Thu May 19 12:20:23 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:20:23.213079 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53df6c4b-0585-4ec0-afc7-cb2b60309315 service cinder] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:20:23.214725 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-53df6c4b-0585-4ec0-afc7-cb2b60309315 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:20:23.293601 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53df6c4b-0585-4ec0-afc7-cb2b60309315 service cinder] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:20:23.298219 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ceb0e4-1112-49a0-ad54-89d524b2690b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:20:23.298886 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ceb0e4-1112-49a0-ad54-89d524b2690b admin admin] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:20:23.299171 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ceb0e4-1112-49a0-ad54-89d524b2690b admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:20:23.299494 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ceb0e4-1112-49a0-ad54-89d524b2690b admin admin] PATH_INFO: `/v3/projects` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:20:23.300223 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ceb0e4-1112-49a0-ad54-89d524b2690b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:20:23.302168 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ceb0e4-1112-49a0-ad54-89d524b2690b admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:20:23.430904 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1115/2209] 10.222.0.23 () {60 vars in 1152 bytes} [Thu May 19 12:20:23 2022] POST /identity/v3/projects => generated 371 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 12:20:23.435143 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a28b443-16c6-4920-9fd6-04c4ce30defd None None] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:20:23.446177 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a28b443-16c6-4920-9fd6-04c4ce30defd None None] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:20:23.504721 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea38b28d-8796-4846-9110-6a6b2dc24a16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:20:23.505332 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea38b28d-8796-4846-9110-6a6b2dc24a16 admin admin] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:20:23.505619 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea38b28d-8796-4846-9110-6a6b2dc24a16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:20:23.505897 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea38b28d-8796-4846-9110-6a6b2dc24a16 admin admin] PATH_INFO: `/v3/users` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:20:23.506610 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea38b28d-8796-4846-9110-6a6b2dc24a16 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:20:23.509172 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea38b28d-8796-4846-9110-6a6b2dc24a16 admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:20:23.644580 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1095/2210] 10.222.0.23 () {60 vars in 1143 bytes} [Thu May 19 12:20:23 2022] POST /identity/v3/users => generated 292 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 12:20:23.646045 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ceb0e4-1112-49a0-ad54-89d524b2690b admin admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:20:23.661184 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ceb0e4-1112-49a0-ad54-89d524b2690b admin admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:20:23.674130 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30219c8e-a859-400f-a4d3-83cd7bc92a91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:20:23.683901 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30219c8e-a859-400f-a4d3-83cd7bc92a91 admin admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:20:23.683901 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30219c8e-a859-400f-a4d3-83cd7bc92a91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:20:23.683901 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30219c8e-a859-400f-a4d3-83cd7bc92a91 admin admin] PATH_INFO: `/v3/roles` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:20:23.684227 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91458293-a1f8-46c7-813d-4e4609b46ee0 None None] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:20:23.684304 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30219c8e-a859-400f-a4d3-83cd7bc92a91 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:20:23.684373 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91458293-a1f8-46c7-813d-4e4609b46ee0 None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:20:23.684373 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91458293-a1f8-46c7-813d-4e4609b46ee0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:20:23.684508 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30219c8e-a859-400f-a4d3-83cd7bc92a91 admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:20:23.695479 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1116/2211] 10.222.0.23 () {58 vars in 1121 bytes} [Thu May 19 12:20:23 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 12:20:23.696732 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30219c8e-a859-400f-a4d3-83cd7bc92a91 admin admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:20:23.701811 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.auth.core [None req-91458293-a1f8-46c7-813d-4e4609b46ee0 None None] MFA Rules not processed for user `8e445569dc5641ed962b0ac977850a6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:20:23.703015 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-91458293-a1f8-46c7-813d-4e4609b46ee0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:20:23.713689 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30219c8e-a859-400f-a4d3-83cd7bc92a91 admin admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:20:23.719533 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca8b4a5-78c8-4ef4-b82f-8603f6b00bbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:20:23.720588 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca8b4a5-78c8-4ef4-b82f-8603f6b00bbf admin admin] REQUEST_METHOD: `PUT` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:20:23.720885 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca8b4a5-78c8-4ef4-b82f-8603f6b00bbf admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:20:23.721171 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca8b4a5-78c8-4ef4-b82f-8603f6b00bbf admin admin] PATH_INFO: `/v3/projects/53014445f44b456d958282a4240ec7fa/users/1b31637fc40446d29250d335943b36d9/roles/9db5671c65d3414ea2682afa2ccf1403` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:20:23.740658 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca8b4a5-78c8-4ef4-b82f-8603f6b00bbf admin admin] RBAC: Authorizing `identity:create_grant(project_id=53014445f44b456d958282a4240ec7fa, user_id=1b31637fc40446d29250d335943b36d9, role_id=9db5671c65d3414ea2682afa2ccf1403)` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:20:23.745442 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca8b4a5-78c8-4ef4-b82f-8603f6b00bbf admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:20:23.783246 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1096/2212] 10.222.0.23 () {58 vars in 957 bytes} [Thu May 19 12:20:23 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:20:23.828873 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1117/2213] 10.222.0.23 () {60 vars in 1482 bytes} [Thu May 19 12:20:23 2022] PUT /identity/v3/projects/53014445f44b456d958282a4240ec7fa/users/1b31637fc40446d29250d335943b36d9/roles/9db5671c65d3414ea2682afa2ccf1403 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:20:23.839007 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91458293-a1f8-46c7-813d-4e4609b46ee0 None None] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:20:23.850613 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91458293-a1f8-46c7-813d-4e4609b46ee0 None None] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:20:23.913234 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fa2702-4ad2-42d1-9c56-64beb62f7ebe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:20:23.913975 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fa2702-4ad2-42d1-9c56-64beb62f7ebe admin admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:20:23.914461 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fa2702-4ad2-42d1-9c56-64beb62f7ebe admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:20:23.914746 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fa2702-4ad2-42d1-9c56-64beb62f7ebe admin admin] PATH_INFO: `/v3/roles` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:20:23.915531 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98fa2702-4ad2-42d1-9c56-64beb62f7ebe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:20:23.917371 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98fa2702-4ad2-42d1-9c56-64beb62f7ebe admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:20:23.930199 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1097/2214] 10.222.0.23 () {58 vars in 1121 bytes} [Thu May 19 12:20:23 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 12:20:23.932478 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:20:23.934505 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca8b4a5-78c8-4ef4-b82f-8603f6b00bbf admin admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:20:23.947244 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca8b4a5-78c8-4ef4-b82f-8603f6b00bbf admin admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:20:23.952410 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f232520-033a-4991-8a0e-cb5939540454 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:20:23.953217 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f232520-033a-4991-8a0e-cb5939540454 admin admin] REQUEST_METHOD: `PUT` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:20:23.953509 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f232520-033a-4991-8a0e-cb5939540454 admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:20:23.953790 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f232520-033a-4991-8a0e-cb5939540454 admin admin] PATH_INFO: `/v3/projects/53014445f44b456d958282a4240ec7fa/users/1b31637fc40446d29250d335943b36d9/roles/9db5671c65d3414ea2682afa2ccf1403` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:20:23.956025 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f232520-033a-4991-8a0e-cb5939540454 admin admin] RBAC: Authorizing `identity:create_grant(project_id=53014445f44b456d958282a4240ec7fa, user_id=1b31637fc40446d29250d335943b36d9, role_id=9db5671c65d3414ea2682afa2ccf1403)` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:20:23.958702 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f232520-033a-4991-8a0e-cb5939540454 admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:20:23.970828 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1118/2215] 10.222.0.23 () {60 vars in 1482 bytes} [Thu May 19 12:20:23 2022] PUT /identity/v3/projects/53014445f44b456d958282a4240ec7fa/users/1b31637fc40446d29250d335943b36d9/roles/9db5671c65d3414ea2682afa2ccf1403 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:20:24.405815 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:20:25.951105 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f689eae4-470b-415f-b9e1-e7c1ceb9e882 None None] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:20:25.951672 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f689eae4-470b-415f-b9e1-e7c1ceb9e882 None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:20:25.951672 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f689eae4-470b-415f-b9e1-e7c1ceb9e882 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:20:25.985835 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.auth.core [None req-f689eae4-470b-415f-b9e1-e7c1ceb9e882 None None] MFA Rules not processed for user `1b31637fc40446d29250d335943b36d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:20:25.987102 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-f689eae4-470b-415f-b9e1-e7c1ceb9e882 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:20:26.068610 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1098/2216] 10.222.0.23 () {58 vars in 957 bytes} [Thu May 19 12:20:25 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:20:26.081869 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f232520-033a-4991-8a0e-cb5939540454 admin admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:20:26.164737 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f232520-033a-4991-8a0e-cb5939540454 admin admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:20:26.169862 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17858787-8f8f-4c91-a9f5-c5219c1182de service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8800254c0e784e42a791fb9abc274d79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd70d25bf69314820b8aaecb65576978a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:20:26.171007 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17858787-8f8f-4c91-a9f5-c5219c1182de service cinder] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:20:26.171523 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17858787-8f8f-4c91-a9f5-c5219c1182de service cinder] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:20:26.172004 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17858787-8f8f-4c91-a9f5-c5219c1182de service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:20:26.240058 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17858787-8f8f-4c91-a9f5-c5219c1182de service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:20:26.241659 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17858787-8f8f-4c91-a9f5-c5219c1182de service cinder] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:20:26.252577 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1119/2217] 10.222.0.23 () {60 vars in 1365 bytes} [Thu May 19 12:20:26 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:20:27.617745 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:20:29.334125 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f689eae4-470b-415f-b9e1-e7c1ceb9e882 None None] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:20:29.335843 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-f689eae4-470b-415f-b9e1-e7c1ceb9e882 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:20:29.348430 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f689eae4-470b-415f-b9e1-e7c1ceb9e882 None None] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:20:29.353105 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ebb00d-c3ca-4f6b-a0c4-18a434bcd488 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8800254c0e784e42a791fb9abc274d79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd70d25bf69314820b8aaecb65576978a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:20:29.354072 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ebb00d-c3ca-4f6b-a0c4-18a434bcd488 service cinder] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:20:29.354276 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ebb00d-c3ca-4f6b-a0c4-18a434bcd488 service cinder] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:20:29.354727 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ebb00d-c3ca-4f6b-a0c4-18a434bcd488 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:20:29.364722 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ebb00d-c3ca-4f6b-a0c4-18a434bcd488 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:20:29.373527 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ebb00d-c3ca-4f6b-a0c4-18a434bcd488 service cinder] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:20:29.377144 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1099/2218] 10.222.0.23 () {60 vars in 1365 bytes} [Thu May 19 12:20:29 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:20:36.714039 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=78,keystone:INSERT=3 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:20:39.039972 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17858787-8f8f-4c91-a9f5-c5219c1182de service cinder] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:20:39.118253 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17858787-8f8f-4c91-a9f5-c5219c1182de service cinder] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:20:39.123564 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b4223c-6a89-43f7-bcd8-dfd85af28316 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:20:39.124547 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b4223c-6a89-43f7-bcd8-dfd85af28316 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:20:39.124828 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b4223c-6a89-43f7-bcd8-dfd85af28316 admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:20:39.125106 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b4223c-6a89-43f7-bcd8-dfd85af28316 admin admin] PATH_INFO: `/v3/users/2064d45822fc43219fc7b2bc6a712f25` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:20:39.126339 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b4223c-6a89-43f7-bcd8-dfd85af28316 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2064d45822fc43219fc7b2bc6a712f25)` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:20:39.128502 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b4223c-6a89-43f7-bcd8-dfd85af28316 admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:20:39.372480 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=64 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:20:39.471848 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-88b4223c-6a89-43f7-bcd8-dfd85af28316 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2064d45822fc43219fc7b2bc6a712f25'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:20:39.485725 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-88b4223c-6a89-43f7-bcd8-dfd85af28316 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2064d45822fc43219fc7b2bc6a712f25'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:20:39.486496 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-88b4223c-6a89-43f7-bcd8-dfd85af28316 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2064d45822fc43219fc7b2bc6a712f25'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:20:39.577042 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-88b4223c-6a89-43f7-bcd8-dfd85af28316 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2064d45822fc43219fc7b2bc6a712f25'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:20:39.585765 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-88b4223c-6a89-43f7-bcd8-dfd85af28316 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2064d45822fc43219fc7b2bc6a712f25'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:20:39.592844 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1120/2219] 10.222.0.23 () {58 vars in 1223 bytes} [Thu May 19 12:20:39 2022] DELETE /identity/v3/users/2064d45822fc43219fc7b2bc6a712f25 => generated 0 bytes in 554 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:20:39.597865 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1942574-9d7e-4084-81d7-a7ac1c96c7d3 None None] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:20:39.598321 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1942574-9d7e-4084-81d7-a7ac1c96c7d3 None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:20:39.598605 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1942574-9d7e-4084-81d7-a7ac1c96c7d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:20:39.618503 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.auth.core [None req-e1942574-9d7e-4084-81d7-a7ac1c96c7d3 None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:20:39.619730 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-e1942574-9d7e-4084-81d7-a7ac1c96c7d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:20:39.684990 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1100/2220] 10.222.0.23 () {58 vars in 957 bytes} [Thu May 19 12:20:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:20:39.700943 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b4223c-6a89-43f7-bcd8-dfd85af28316 admin admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:20:39.702394 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-88b4223c-6a89-43f7-bcd8-dfd85af28316 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:20:39.763617 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b4223c-6a89-43f7-bcd8-dfd85af28316 admin admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:20:39.767362 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62253f35-b59c-4777-b66a-ce1d5e4c2550 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a00b3fa7497456895e5e0bf105c4fc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd70d25bf69314820b8aaecb65576978a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:20:39.768060 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62253f35-b59c-4777-b66a-ce1d5e4c2550 service neutron] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:20:39.768427 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62253f35-b59c-4777-b66a-ce1d5e4c2550 service neutron] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:20:39.768714 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62253f35-b59c-4777-b66a-ce1d5e4c2550 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:20:39.778939 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62253f35-b59c-4777-b66a-ce1d5e4c2550 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:20:39.780590 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62253f35-b59c-4777-b66a-ce1d5e4c2550 service neutron] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:20:39.791256 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1121/2221] 10.222.0.23 () {60 vars in 1365 bytes} [Thu May 19 12:20:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:20:40.346756 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1942574-9d7e-4084-81d7-a7ac1c96c7d3 None None] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:20:40.348319 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-e1942574-9d7e-4084-81d7-a7ac1c96c7d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:20:40.362303 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1942574-9d7e-4084-81d7-a7ac1c96c7d3 None None] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:20:40.367772 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930a1ce0-68c4-42ed-b956-9cb59442f096 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:20:40.368833 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930a1ce0-68c4-42ed-b956-9cb59442f096 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:20:40.369457 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930a1ce0-68c4-42ed-b956-9cb59442f096 admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:20:40.369893 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930a1ce0-68c4-42ed-b956-9cb59442f096 admin admin] PATH_INFO: `/v3/projects/776e5ee8d7bf4f3c9cc37b6e30b5cd02` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:20:40.371177 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930a1ce0-68c4-42ed-b956-9cb59442f096 admin admin] RBAC: Authorizing `identity:delete_project(project_id=776e5ee8d7bf4f3c9cc37b6e30b5cd02)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:20:40.373339 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930a1ce0-68c4-42ed-b956-9cb59442f096 admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:20:40.557239 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-930a1ce0-68c4-42ed-b956-9cb59442f096 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '776e5ee8d7bf4f3c9cc37b6e30b5cd02', 'request_id': 'req-930a1ce0-68c4-42ed-b956-9cb59442f096'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:20:40.566540 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-930a1ce0-68c4-42ed-b956-9cb59442f096 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '776e5ee8d7bf4f3c9cc37b6e30b5cd02', 'request_id': 'req-930a1ce0-68c4-42ed-b956-9cb59442f096'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:20:40.725241 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-930a1ce0-68c4-42ed-b956-9cb59442f096 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '776e5ee8d7bf4f3c9cc37b6e30b5cd02', 'request_id': 'req-930a1ce0-68c4-42ed-b956-9cb59442f096'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:20:40.738072 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-930a1ce0-68c4-42ed-b956-9cb59442f096 admin admin] The token cache is being invalidate because project 776e5ee8d7bf4f3c9cc37b6e30b5cd02 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80539) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 12:20:40.738357 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-930a1ce0-68c4-42ed-b956-9cb59442f096 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:20:40.739012 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-930a1ce0-68c4-42ed-b956-9cb59442f096 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:20:40.740593 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1101/2222] 10.222.0.23 () {58 vars in 1232 bytes} [Thu May 19 12:20:40 2022] DELETE /identity/v3/projects/776e5ee8d7bf4f3c9cc37b6e30b5cd02 => generated 0 bytes in 394 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:20:40.746049 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e22bb07-1912-4050-9080-44469ca3234c None None] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:20:40.746617 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e22bb07-1912-4050-9080-44469ca3234c None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:20:40.746617 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e22bb07-1912-4050-9080-44469ca3234c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:20:40.766431 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.auth.core [None req-5e22bb07-1912-4050-9080-44469ca3234c None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:20:40.767654 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-5e22bb07-1912-4050-9080-44469ca3234c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:20:40.788970 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:20:40.850694 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1122/2223] 10.222.0.23 () {58 vars in 957 bytes} [Thu May 19 12:20:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 12:20:40.860068 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f52765-ec6e-4bf4-b3f2-79d7b8041fd6 None None] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:20:40.860358 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f52765-ec6e-4bf4-b3f2-79d7b8041fd6 None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:20:40.860999 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f52765-ec6e-4bf4-b3f2-79d7b8041fd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:20:40.885144 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.auth.core [None req-c7f52765-ec6e-4bf4-b3f2-79d7b8041fd6 None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:20:40.886433 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-c7f52765-ec6e-4bf4-b3f2-79d7b8041fd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:20:40.896986 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1102/2224] 10.222.0.23 () {58 vars in 957 bytes} [Thu May 19 12:20:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:20:40.899648 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e22bb07-1912-4050-9080-44469ca3234c None None] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:20:40.909681 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e22bb07-1912-4050-9080-44469ca3234c None None] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:20:40.916331 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae006a65-276d-431a-944e-f5c490120b00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:20:40.917072 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae006a65-276d-431a-944e-f5c490120b00 admin admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:20:40.919599 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae006a65-276d-431a-944e-f5c490120b00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:20:40.919599 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae006a65-276d-431a-944e-f5c490120b00 admin admin] PATH_INFO: `/v3/domains` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:20:40.919599 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae006a65-276d-431a-944e-f5c490120b00 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:20:40.919599 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae006a65-276d-431a-944e-f5c490120b00 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:20:40.920451 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae006a65-276d-431a-944e-f5c490120b00 admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:20:40.938490 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1123/2225] 10.222.0.23 () {58 vars in 1152 bytes} [Thu May 19 12:20:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:20:40.940441 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f52765-ec6e-4bf4-b3f2-79d7b8041fd6 None None] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:20:40.951753 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f52765-ec6e-4bf4-b3f2-79d7b8041fd6 None None] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:20:40.958628 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9539e8d9-74ca-40ae-a491-5052585a2787 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:20:40.959536 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9539e8d9-74ca-40ae-a491-5052585a2787 admin admin] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:20:40.959814 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9539e8d9-74ca-40ae-a491-5052585a2787 admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:20:40.960276 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9539e8d9-74ca-40ae-a491-5052585a2787 admin admin] PATH_INFO: `/v3/projects` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:20:40.962731 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9539e8d9-74ca-40ae-a491-5052585a2787 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:20:40.978686 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9539e8d9-74ca-40ae-a491-5052585a2787 admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:20:41.168977 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1103/2226] 10.222.0.23 () {60 vars in 1152 bytes} [Thu May 19 12:20:40 2022] POST /identity/v3/projects => generated 371 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 12:20:41.173804 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae006a65-276d-431a-944e-f5c490120b00 admin admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:20:41.185816 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae006a65-276d-431a-944e-f5c490120b00 admin admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:20:41.255579 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2561b503-6f79-4bb7-a3a8-ee9cb46ecdee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:20:41.256826 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2561b503-6f79-4bb7-a3a8-ee9cb46ecdee admin admin] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:20:41.257209 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2561b503-6f79-4bb7-a3a8-ee9cb46ecdee admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:20:41.258133 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2561b503-6f79-4bb7-a3a8-ee9cb46ecdee admin admin] PATH_INFO: `/v3/users` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:20:41.259451 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2561b503-6f79-4bb7-a3a8-ee9cb46ecdee admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:20:41.263329 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2561b503-6f79-4bb7-a3a8-ee9cb46ecdee admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:20:41.457349 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1124/2227] 10.222.0.23 () {60 vars in 1143 bytes} [Thu May 19 12:20:41 2022] POST /identity/v3/users => generated 286 bytes in 284 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 12:20:41.461580 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9539e8d9-74ca-40ae-a491-5052585a2787 admin admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:20:41.475892 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9539e8d9-74ca-40ae-a491-5052585a2787 admin admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:20:41.482480 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ebcc97-e5a4-4df7-be7d-60cdd7406248 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:20:41.483356 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ebcc97-e5a4-4df7-be7d-60cdd7406248 admin admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:20:41.483738 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ebcc97-e5a4-4df7-be7d-60cdd7406248 admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:20:41.484382 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ebcc97-e5a4-4df7-be7d-60cdd7406248 admin admin] PATH_INFO: `/v3/roles` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:20:41.485087 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ebcc97-e5a4-4df7-be7d-60cdd7406248 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:20:41.486856 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ebcc97-e5a4-4df7-be7d-60cdd7406248 admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:20:41.502598 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1104/2228] 10.222.0.23 () {58 vars in 1121 bytes} [Thu May 19 12:20:41 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 12:20:41.503809 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2561b503-6f79-4bb7-a3a8-ee9cb46ecdee admin admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:20:41.515406 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2561b503-6f79-4bb7-a3a8-ee9cb46ecdee admin admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:20:41.521911 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5442ddf2-ebba-4d61-95d2-2c9c8d6a46c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:20:41.522620 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5442ddf2-ebba-4d61-95d2-2c9c8d6a46c6 admin admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:20:41.522941 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5442ddf2-ebba-4d61-95d2-2c9c8d6a46c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:20:41.523953 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5442ddf2-ebba-4d61-95d2-2c9c8d6a46c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:20:41.527375 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5442ddf2-ebba-4d61-95d2-2c9c8d6a46c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:20:41.529136 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5442ddf2-ebba-4d61-95d2-2c9c8d6a46c6 admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:20:41.541297 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1125/2229] 10.222.0.23 () {58 vars in 1121 bytes} [Thu May 19 12:20:41 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 19 12:20:41.549048 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ebcc97-e5a4-4df7-be7d-60cdd7406248 admin admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:20:41.559766 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ebcc97-e5a4-4df7-be7d-60cdd7406248 admin admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:20:41.564538 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d59054-3219-4522-a283-629e8dfa82b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:20:41.565444 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d59054-3219-4522-a283-629e8dfa82b5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:20:41.565725 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d59054-3219-4522-a283-629e8dfa82b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:20:41.566004 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d59054-3219-4522-a283-629e8dfa82b5 admin admin] PATH_INFO: `/v3/projects/0beba4cdb9854497b6cedd3e76059e12/users/3124804c65204e9e8dc4c26e572390b9/roles/259eb400cdb4439db72d14456ae9cd08` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:20:41.584672 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d59054-3219-4522-a283-629e8dfa82b5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0beba4cdb9854497b6cedd3e76059e12, user_id=3124804c65204e9e8dc4c26e572390b9, role_id=259eb400cdb4439db72d14456ae9cd08)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:20:41.588548 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d59054-3219-4522-a283-629e8dfa82b5 admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:20:41.743378 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1105/2230] 10.222.0.23 () {60 vars in 1482 bytes} [Thu May 19 12:20:41 2022] PUT /identity/v3/projects/0beba4cdb9854497b6cedd3e76059e12/users/3124804c65204e9e8dc4c26e572390b9/roles/259eb400cdb4439db72d14456ae9cd08 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:20:41.753005 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16474021-1f55-45f7-91c7-c8152810329b None None] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:20:41.755873 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16474021-1f55-45f7-91c7-c8152810329b None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:20:41.755873 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16474021-1f55-45f7-91c7-c8152810329b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:20:41.786816 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.auth.core [None req-16474021-1f55-45f7-91c7-c8152810329b None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:20:41.788123 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-16474021-1f55-45f7-91c7-c8152810329b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:20:41.860043 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1126/2231] 10.222.0.23 () {58 vars in 957 bytes} [Thu May 19 12:20:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:20:41.873103 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d59054-3219-4522-a283-629e8dfa82b5 admin admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:20:41.874852 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-09d59054-3219-4522-a283-629e8dfa82b5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:20:41.903992 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:20:41.957651 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d59054-3219-4522-a283-629e8dfa82b5 admin admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:20:41.963725 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4e7fc5-bcc0-4d40-bef8-a00719d6baee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a00b3fa7497456895e5e0bf105c4fc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd70d25bf69314820b8aaecb65576978a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:20:41.973998 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4e7fc5-bcc0-4d40-bef8-a00719d6baee service neutron] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:20:41.973998 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4e7fc5-bcc0-4d40-bef8-a00719d6baee service neutron] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:20:41.973998 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4e7fc5-bcc0-4d40-bef8-a00719d6baee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:20:41.978357 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4e7fc5-bcc0-4d40-bef8-a00719d6baee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:20:41.979842 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4e7fc5-bcc0-4d40-bef8-a00719d6baee service neutron] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:20:41.991287 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1106/2232] 10.222.0.23 () {60 vars in 1365 bytes} [Thu May 19 12:20:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 12:20:50.056807 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfc05bf-227e-4710-bf98-42794b80a192 None None] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:20:50.058984 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfc05bf-227e-4710-bf98-42794b80a192 None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:20:50.059652 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfc05bf-227e-4710-bf98-42794b80a192 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:20:50.091541 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.auth.core [None req-6bfc05bf-227e-4710-bf98-42794b80a192 None None] MFA Rules not processed for user `3124804c65204e9e8dc4c26e572390b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:20:50.092932 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-6bfc05bf-227e-4710-bf98-42794b80a192 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:20:50.116686 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:20:50.185023 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1127/2233] 10.222.0.23 () {58 vars in 957 bytes} [Thu May 19 12:20:50 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:20:50.200239 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4e7fc5-bcc0-4d40-bef8-a00719d6baee service neutron] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:20:50.211435 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4e7fc5-bcc0-4d40-bef8-a00719d6baee service neutron] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:20:50.217065 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be810d9d-7d2e-4140-aefb-e25ae3bd62a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:20:50.218093 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be810d9d-7d2e-4140-aefb-e25ae3bd62a6 admin admin] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:20:50.218443 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be810d9d-7d2e-4140-aefb-e25ae3bd62a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:20:50.218940 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be810d9d-7d2e-4140-aefb-e25ae3bd62a6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:20:50.219967 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be810d9d-7d2e-4140-aefb-e25ae3bd62a6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:20:50.222402 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be810d9d-7d2e-4140-aefb-e25ae3bd62a6 admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:20:50.342428 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1107/2234] 10.222.0.23 () {60 vars in 1152 bytes} [Thu May 19 12:20:50 2022] POST /identity/v3/projects => generated 367 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 12:20:50.343712 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfc05bf-227e-4710-bf98-42794b80a192 None None] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:20:50.355655 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfc05bf-227e-4710-bf98-42794b80a192 None None] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:20:50.423266 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f5cdf4-5fcf-4efd-9ed3-516e1265f4f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:20:50.424016 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f5cdf4-5fcf-4efd-9ed3-516e1265f4f1 admin admin] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:20:50.424305 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f5cdf4-5fcf-4efd-9ed3-516e1265f4f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:20:50.424585 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f5cdf4-5fcf-4efd-9ed3-516e1265f4f1 admin admin] PATH_INFO: `/v3/users` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:20:50.425305 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f5cdf4-5fcf-4efd-9ed3-516e1265f4f1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:20:50.427507 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f5cdf4-5fcf-4efd-9ed3-516e1265f4f1 admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:20:50.515019 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1128/2235] 10.222.0.23 () {60 vars in 1143 bytes} [Thu May 19 12:20:50 2022] POST /identity/v3/users => generated 290 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 12:20:50.519041 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be810d9d-7d2e-4140-aefb-e25ae3bd62a6 admin admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:20:50.531111 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be810d9d-7d2e-4140-aefb-e25ae3bd62a6 admin admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:20:50.535896 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92574ab6-fb91-4fcd-a4a9-6f3722865299 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:20:50.536570 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92574ab6-fb91-4fcd-a4a9-6f3722865299 admin admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:20:50.536858 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92574ab6-fb91-4fcd-a4a9-6f3722865299 admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:20:50.537748 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92574ab6-fb91-4fcd-a4a9-6f3722865299 admin admin] PATH_INFO: `/v3/roles` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:20:50.538692 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92574ab6-fb91-4fcd-a4a9-6f3722865299 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:20:50.541657 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92574ab6-fb91-4fcd-a4a9-6f3722865299 admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:20:50.553620 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1108/2236] 10.222.0.23 () {58 vars in 1121 bytes} [Thu May 19 12:20:50 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 12:20:50.558038 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f5cdf4-5fcf-4efd-9ed3-516e1265f4f1 admin admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:20:50.568918 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f5cdf4-5fcf-4efd-9ed3-516e1265f4f1 admin admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:20:50.574177 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0bf111b-d169-4f50-b14b-832830c17d10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:20:50.575086 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bf111b-d169-4f50-b14b-832830c17d10 admin admin] REQUEST_METHOD: `PUT` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:20:50.575365 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bf111b-d169-4f50-b14b-832830c17d10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:20:50.575637 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bf111b-d169-4f50-b14b-832830c17d10 admin admin] PATH_INFO: `/v3/projects/03e87054b75946338efbef7540c97760/users/82b6198e186449c5bae4c6c41ed2d813/roles/9db5671c65d3414ea2682afa2ccf1403` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:20:50.591382 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0bf111b-d169-4f50-b14b-832830c17d10 admin admin] RBAC: Authorizing `identity:create_grant(project_id=03e87054b75946338efbef7540c97760, user_id=82b6198e186449c5bae4c6c41ed2d813, role_id=9db5671c65d3414ea2682afa2ccf1403)` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:20:50.594193 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0bf111b-d169-4f50-b14b-832830c17d10 admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:20:50.674340 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1129/2237] 10.222.0.23 () {60 vars in 1482 bytes} [Thu May 19 12:20:50 2022] PUT /identity/v3/projects/03e87054b75946338efbef7540c97760/users/82b6198e186449c5bae4c6c41ed2d813/roles/9db5671c65d3414ea2682afa2ccf1403 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:20:50.678419 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92574ab6-fb91-4fcd-a4a9-6f3722865299 admin admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:20:50.689295 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92574ab6-fb91-4fcd-a4a9-6f3722865299 admin admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:20:50.752466 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ccc89c5-2781-437a-894b-e5253158d3e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:20:50.753141 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccc89c5-2781-437a-894b-e5253158d3e6 admin admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:20:50.753427 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccc89c5-2781-437a-894b-e5253158d3e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:20:50.753708 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccc89c5-2781-437a-894b-e5253158d3e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:20:50.754444 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ccc89c5-2781-437a-894b-e5253158d3e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:20:50.756163 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ccc89c5-2781-437a-894b-e5253158d3e6 admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:20:50.767778 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1109/2238] 10.222.0.23 () {58 vars in 1121 bytes} [Thu May 19 12:20:50 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 12:20:50.770872 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0bf111b-d169-4f50-b14b-832830c17d10 admin admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:20:50.782226 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0bf111b-d169-4f50-b14b-832830c17d10 admin admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:20:50.787237 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe7df27-4801-4483-8a3c-ee450521e034 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:20:50.788156 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe7df27-4801-4483-8a3c-ee450521e034 admin admin] REQUEST_METHOD: `PUT` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:20:50.788467 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe7df27-4801-4483-8a3c-ee450521e034 admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:20:50.789415 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe7df27-4801-4483-8a3c-ee450521e034 admin admin] PATH_INFO: `/v3/projects/03e87054b75946338efbef7540c97760/users/82b6198e186449c5bae4c6c41ed2d813/roles/9db5671c65d3414ea2682afa2ccf1403` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:20:50.791433 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe7df27-4801-4483-8a3c-ee450521e034 admin admin] RBAC: Authorizing `identity:create_grant(project_id=03e87054b75946338efbef7540c97760, user_id=82b6198e186449c5bae4c6c41ed2d813, role_id=9db5671c65d3414ea2682afa2ccf1403)` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:20:50.794322 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe7df27-4801-4483-8a3c-ee450521e034 admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:20:50.805748 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1130/2239] 10.222.0.23 () {60 vars in 1482 bytes} [Thu May 19 12:20:50 2022] PUT /identity/v3/projects/03e87054b75946338efbef7540c97760/users/82b6198e186449c5bae4c6c41ed2d813/roles/9db5671c65d3414ea2682afa2ccf1403 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:20:55.126123 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbedb5ce-65c4-4af8-87a1-60d308e90e5b None None] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:20:55.126686 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbedb5ce-65c4-4af8-87a1-60d308e90e5b None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:20:55.126686 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbedb5ce-65c4-4af8-87a1-60d308e90e5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:20:55.262141 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.auth.core [None req-dbedb5ce-65c4-4af8-87a1-60d308e90e5b None None] MFA Rules not processed for user `0c3181fb4f584a549a786adfefdf3dfb`. Rule list: `[]` (Enabled: `True`). {{(pid=80539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:20:55.264741 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-dbedb5ce-65c4-4af8-87a1-60d308e90e5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:20:55.393163 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:20:55.444597 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1110/2240] 10.222.0.23 () {58 vars in 1009 bytes} [Thu May 19 12:20:55 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 315 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:20:55.475194 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe7df27-4801-4483-8a3c-ee450521e034 admin admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:20:55.478507 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-afe7df27-4801-4483-8a3c-ee450521e034 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:20:55.739384 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:20:55.783636 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe7df27-4801-4483-8a3c-ee450521e034 admin admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:20:55.791869 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f5f65a-1c9d-4998-b701-50cc1c738617 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15cbfc73ae1243d2a8b1e70f9cd99876', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd70d25bf69314820b8aaecb65576978a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:20:55.792685 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f5f65a-1c9d-4998-b701-50cc1c738617 service swift] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:20:55.792972 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f5f65a-1c9d-4998-b701-50cc1c738617 service swift] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:20:55.793311 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f5f65a-1c9d-4998-b701-50cc1c738617 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:20:55.805437 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f5f65a-1c9d-4998-b701-50cc1c738617 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:20:55.807286 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f5f65a-1c9d-4998-b701-50cc1c738617 service swift] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:20:55.817304 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1131/2241] 10.222.0.23 () {60 vars in 1384 bytes} [Thu May 19 12:20:55 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 19 12:20:58.022675 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbedb5ce-65c4-4af8-87a1-60d308e90e5b None None] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:20:58.029145 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-dbedb5ce-65c4-4af8-87a1-60d308e90e5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:20:58.144945 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbedb5ce-65c4-4af8-87a1-60d308e90e5b None None] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:20:58.155955 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261c6303-bb02-4e95-ab0d-b8428b320efd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:20:58.157175 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261c6303-bb02-4e95-ab0d-b8428b320efd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:20:58.157621 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261c6303-bb02-4e95-ab0d-b8428b320efd admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:20:58.158315 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261c6303-bb02-4e95-ab0d-b8428b320efd admin admin] PATH_INFO: `/v3/users/5db46f6216f347de9816b2227cbecae7` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:20:58.159843 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261c6303-bb02-4e95-ab0d-b8428b320efd admin admin] RBAC: Authorizing `identity:delete_user(user_id=5db46f6216f347de9816b2227cbecae7)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:20:58.162013 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261c6303-bb02-4e95-ab0d-b8428b320efd admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:20:58.472721 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-261c6303-bb02-4e95-ab0d-b8428b320efd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5db46f6216f347de9816b2227cbecae7'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:20:58.502987 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-261c6303-bb02-4e95-ab0d-b8428b320efd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5db46f6216f347de9816b2227cbecae7'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:20:58.504390 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-261c6303-bb02-4e95-ab0d-b8428b320efd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5db46f6216f347de9816b2227cbecae7'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:20:58.595106 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b79368-cad7-49bb-ac99-a2f467ca5d1b None None] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:20:58.595640 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b79368-cad7-49bb-ac99-a2f467ca5d1b None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:20:58.595889 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b79368-cad7-49bb-ac99-a2f467ca5d1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:20:58.628323 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-261c6303-bb02-4e95-ab0d-b8428b320efd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5db46f6216f347de9816b2227cbecae7'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:20:58.647129 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-261c6303-bb02-4e95-ab0d-b8428b320efd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5db46f6216f347de9816b2227cbecae7'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:20:58.669952 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1111/2242] 10.222.0.23 () {58 vars in 1223 bytes} [Thu May 19 12:20:58 2022] DELETE /identity/v3/users/5db46f6216f347de9816b2227cbecae7 => generated 0 bytes in 647 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 12:20:58.691560 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.auth.core [None req-11b79368-cad7-49bb-ac99-a2f467ca5d1b None None] MFA Rules not processed for user `82b6198e186449c5bae4c6c41ed2d813`. Rule list: `[]` (Enabled: `True`). {{(pid=80540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:20:58.693712 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-11b79368-cad7-49bb-ac99-a2f467ca5d1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:20:58.850404 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1132/2243] 10.222.0.23 () {58 vars in 957 bytes} [Thu May 19 12:20:58 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 257 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:20:58.870174 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261c6303-bb02-4e95-ab0d-b8428b320efd admin admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:20:58.872202 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-261c6303-bb02-4e95-ab0d-b8428b320efd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:20:59.033386 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261c6303-bb02-4e95-ab0d-b8428b320efd admin admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:20:59.045199 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3caa65cc-775b-4f83-a6c3-8e07d9a7f94f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34a98d75d84b4c67981377e0f26eca62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd70d25bf69314820b8aaecb65576978a', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:20:59.046046 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3caa65cc-775b-4f83-a6c3-8e07d9a7f94f service nova] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:20:59.046400 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3caa65cc-775b-4f83-a6c3-8e07d9a7f94f service nova] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:20:59.046729 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3caa65cc-775b-4f83-a6c3-8e07d9a7f94f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:20:59.048761 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-3caa65cc-775b-4f83-a6c3-8e07d9a7f94f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:20:59.054691 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:20:59.082177 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b79368-cad7-49bb-ac99-a2f467ca5d1b None None] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:20:59.089833 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-11b79368-cad7-49bb-ac99-a2f467ca5d1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:20:59.216332 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3caa65cc-775b-4f83-a6c3-8e07d9a7f94f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:20:59.220652 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3caa65cc-775b-4f83-a6c3-8e07d9a7f94f service nova] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:20:59.234602 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1112/2244] 10.222.0.23 () {60 vars in 1365 bytes} [Thu May 19 12:20:58 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:20:59.259986 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b79368-cad7-49bb-ac99-a2f467ca5d1b None None] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:20:59.266386 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38204a64-1e1c-457f-aa0e-9f23a31cebee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:20:59.267911 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38204a64-1e1c-457f-aa0e-9f23a31cebee admin admin] REQUEST_METHOD: `DELETE` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:20:59.268445 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38204a64-1e1c-457f-aa0e-9f23a31cebee admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:20:59.268923 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38204a64-1e1c-457f-aa0e-9f23a31cebee admin admin] PATH_INFO: `/v3/projects/39be464eb5ec429aa888dd81bbe31496` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:20:59.271181 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38204a64-1e1c-457f-aa0e-9f23a31cebee admin admin] RBAC: Authorizing `identity:delete_project(project_id=39be464eb5ec429aa888dd81bbe31496)` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:20:59.274931 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38204a64-1e1c-457f-aa0e-9f23a31cebee admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:20:59.428221 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-38204a64-1e1c-457f-aa0e-9f23a31cebee admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '39be464eb5ec429aa888dd81bbe31496', 'request_id': 'req-38204a64-1e1c-457f-aa0e-9f23a31cebee'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:20:59.446975 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-38204a64-1e1c-457f-aa0e-9f23a31cebee admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '39be464eb5ec429aa888dd81bbe31496', 'request_id': 'req-38204a64-1e1c-457f-aa0e-9f23a31cebee'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:20:59.456513 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-38204a64-1e1c-457f-aa0e-9f23a31cebee admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '39be464eb5ec429aa888dd81bbe31496', 'request_id': 'req-38204a64-1e1c-457f-aa0e-9f23a31cebee'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:20:59.461454 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:20:59.672909 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-38204a64-1e1c-457f-aa0e-9f23a31cebee admin admin] The token cache is being invalidate because project 39be464eb5ec429aa888dd81bbe31496 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80540) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 12:20:59.697110 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-38204a64-1e1c-457f-aa0e-9f23a31cebee admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:20:59.697110 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-38204a64-1e1c-457f-aa0e-9f23a31cebee admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:20:59.697110 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1133/2245] 10.222.0.23 () {58 vars in 1232 bytes} [Thu May 19 12:20:59 2022] DELETE /identity/v3/projects/39be464eb5ec429aa888dd81bbe31496 => generated 0 bytes in 600 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:20:59.698317 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3caa65cc-775b-4f83-a6c3-8e07d9a7f94f service nova] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:20:59.702418 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3caa65cc-775b-4f83-a6c3-8e07d9a7f94f service nova] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:20:59.871691 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9866a3bb-086f-4a5a-a5d3-3c4367821116 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:20:59.872618 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9866a3bb-086f-4a5a-a5d3-3c4367821116 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:20:59.872908 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9866a3bb-086f-4a5a-a5d3-3c4367821116 admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:20:59.873198 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9866a3bb-086f-4a5a-a5d3-3c4367821116 admin admin] PATH_INFO: `/v3/users/1b31637fc40446d29250d335943b36d9` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:20:59.874481 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9866a3bb-086f-4a5a-a5d3-3c4367821116 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1b31637fc40446d29250d335943b36d9)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:20:59.880762 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9866a3bb-086f-4a5a-a5d3-3c4367821116 admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:21:00.419181 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-9866a3bb-086f-4a5a-a5d3-3c4367821116 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1b31637fc40446d29250d335943b36d9'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:21:00.443102 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-9866a3bb-086f-4a5a-a5d3-3c4367821116 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1b31637fc40446d29250d335943b36d9'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:21:00.444744 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-9866a3bb-086f-4a5a-a5d3-3c4367821116 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1b31637fc40446d29250d335943b36d9'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:21:00.588976 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-9866a3bb-086f-4a5a-a5d3-3c4367821116 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1b31637fc40446d29250d335943b36d9'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:21:00.605785 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-9866a3bb-086f-4a5a-a5d3-3c4367821116 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1b31637fc40446d29250d335943b36d9'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:21:00.624952 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1113/2246] 10.222.0.23 () {58 vars in 1223 bytes} [Thu May 19 12:20:59 2022] DELETE /identity/v3/users/1b31637fc40446d29250d335943b36d9 => generated 0 bytes in 942 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:21:01.019778 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38204a64-1e1c-457f-aa0e-9f23a31cebee admin admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:21:01.022437 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-38204a64-1e1c-457f-aa0e-9f23a31cebee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:21:01.053162 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38204a64-1e1c-457f-aa0e-9f23a31cebee admin admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:21:01.199342 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2330e6cc-48dc-4845-8596-683e3e4d38c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:21:01.200501 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2330e6cc-48dc-4845-8596-683e3e4d38c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:21:01.201027 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2330e6cc-48dc-4845-8596-683e3e4d38c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:21:01.201717 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2330e6cc-48dc-4845-8596-683e3e4d38c8 admin admin] PATH_INFO: `/v3/projects/53014445f44b456d958282a4240ec7fa` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:21:01.203071 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2330e6cc-48dc-4845-8596-683e3e4d38c8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=53014445f44b456d958282a4240ec7fa)` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:21:01.205531 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2330e6cc-48dc-4845-8596-683e3e4d38c8 admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:21:01.491090 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-2330e6cc-48dc-4845-8596-683e3e4d38c8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '53014445f44b456d958282a4240ec7fa', 'request_id': 'req-2330e6cc-48dc-4845-8596-683e3e4d38c8'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:21:01.508426 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-2330e6cc-48dc-4845-8596-683e3e4d38c8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '53014445f44b456d958282a4240ec7fa', 'request_id': 'req-2330e6cc-48dc-4845-8596-683e3e4d38c8'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:21:01.525341 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-2330e6cc-48dc-4845-8596-683e3e4d38c8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '53014445f44b456d958282a4240ec7fa', 'request_id': 'req-2330e6cc-48dc-4845-8596-683e3e4d38c8'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:21:01.690086 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-2330e6cc-48dc-4845-8596-683e3e4d38c8 admin admin] The token cache is being invalidate because project 53014445f44b456d958282a4240ec7fa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80540) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 12:21:01.690086 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-2330e6cc-48dc-4845-8596-683e3e4d38c8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:21:01.690086 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-2330e6cc-48dc-4845-8596-683e3e4d38c8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:21:01.690086 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1134/2247] 10.222.0.23 () {58 vars in 1232 bytes} [Thu May 19 12:21:01 2022] DELETE /identity/v3/projects/53014445f44b456d958282a4240ec7fa => generated 0 bytes in 670 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:21:04.020522 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fd03c6-3bab-4ebc-8d4a-824841d2c20a None None] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:21:04.021061 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fd03c6-3bab-4ebc-8d4a-824841d2c20a None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:21:04.021061 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fd03c6-3bab-4ebc-8d4a-824841d2c20a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:21:04.044640 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.auth.core [None req-70fd03c6-3bab-4ebc-8d4a-824841d2c20a None None] MFA Rules not processed for user `3124804c65204e9e8dc4c26e572390b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:21:04.047054 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-70fd03c6-3bab-4ebc-8d4a-824841d2c20a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:21:04.092421 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:21:04.190118 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1114/2248] 10.222.0.23 () {58 vars in 957 bytes} [Thu May 19 12:21:04 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:21:11.681180 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=53,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:21:14.365100 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=14 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:21:19.907634 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2425ead8-5452-43e3-8994-0f4730998a70 None None] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:21:19.908171 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2425ead8-5452-43e3-8994-0f4730998a70 None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:21:19.908296 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2425ead8-5452-43e3-8994-0f4730998a70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:21:19.928664 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.auth.core [None req-2425ead8-5452-43e3-8994-0f4730998a70 None None] MFA Rules not processed for user `0c3181fb4f584a549a786adfefdf3dfb`. Rule list: `[]` (Enabled: `True`). {{(pid=80540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:21:19.930588 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-2425ead8-5452-43e3-8994-0f4730998a70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:21:19.990392 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1135/2249] 10.222.0.23 () {58 vars in 967 bytes} [Thu May 19 12:21:19 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:21:20.003416 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70fd03c6-3bab-4ebc-8d4a-824841d2c20a None None] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:21:20.004963 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-70fd03c6-3bab-4ebc-8d4a-824841d2c20a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:21:20.078576 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70fd03c6-3bab-4ebc-8d4a-824841d2c20a None None] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:21:20.083511 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa198e2-f2bc-4cb4-919e-e22e84baac85 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15cbfc73ae1243d2a8b1e70f9cd99876', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd70d25bf69314820b8aaecb65576978a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:21:20.084190 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa198e2-f2bc-4cb4-919e-e22e84baac85 service swift] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:21:20.084479 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa198e2-f2bc-4cb4-919e-e22e84baac85 service swift] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:21:20.084758 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa198e2-f2bc-4cb4-919e-e22e84baac85 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:21:20.093610 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa198e2-f2bc-4cb4-919e-e22e84baac85 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:21:20.095196 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa198e2-f2bc-4cb4-919e-e22e84baac85 service swift] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:21:20.104221 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1115/2250] 10.222.0.23 () {60 vars in 1384 bytes} [Thu May 19 12:21:20 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 19 12:21:23.838750 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2425ead8-5452-43e3-8994-0f4730998a70 None None] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:21:23.840298 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-2425ead8-5452-43e3-8994-0f4730998a70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:21:23.910513 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2425ead8-5452-43e3-8994-0f4730998a70 None None] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:21:23.915988 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f679c4a-8f31-4b6a-9cf1-6a3293109812 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:21:23.916866 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f679c4a-8f31-4b6a-9cf1-6a3293109812 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:21:23.917152 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f679c4a-8f31-4b6a-9cf1-6a3293109812 admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:21:23.917429 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f679c4a-8f31-4b6a-9cf1-6a3293109812 admin admin] PATH_INFO: `/v3/users/858ef1572f064c248592e92625b228f8` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:21:23.918633 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f679c4a-8f31-4b6a-9cf1-6a3293109812 admin admin] RBAC: Authorizing `identity:delete_user(user_id=858ef1572f064c248592e92625b228f8)` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:21:23.920595 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f679c4a-8f31-4b6a-9cf1-6a3293109812 admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:21:24.145303 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-9f679c4a-8f31-4b6a-9cf1-6a3293109812 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '858ef1572f064c248592e92625b228f8'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:21:24.158172 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-9f679c4a-8f31-4b6a-9cf1-6a3293109812 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '858ef1572f064c248592e92625b228f8'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:21:24.158903 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-9f679c4a-8f31-4b6a-9cf1-6a3293109812 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '858ef1572f064c248592e92625b228f8'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:21:24.251269 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-9f679c4a-8f31-4b6a-9cf1-6a3293109812 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '858ef1572f064c248592e92625b228f8'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:21:24.258908 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-9f679c4a-8f31-4b6a-9cf1-6a3293109812 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '858ef1572f064c248592e92625b228f8'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:21:24.265502 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1136/2251] 10.222.0.23 () {58 vars in 1223 bytes} [Thu May 19 12:21:23 2022] DELETE /identity/v3/users/858ef1572f064c248592e92625b228f8 => generated 0 bytes in 427 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 12:21:24.550295 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa198e2-f2bc-4cb4-919e-e22e84baac85 service swift] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:21:24.563053 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa198e2-f2bc-4cb4-919e-e22e84baac85 service swift] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:21:24.636692 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18a7fb0-f294-4939-b5c4-e284415f5be3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:21:24.637611 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18a7fb0-f294-4939-b5c4-e284415f5be3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:21:24.638061 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18a7fb0-f294-4939-b5c4-e284415f5be3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:21:24.638372 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18a7fb0-f294-4939-b5c4-e284415f5be3 admin admin] PATH_INFO: `/v3/projects/79e70c946e5440e39ee1d7cbeb0a4160` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:21:24.639653 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18a7fb0-f294-4939-b5c4-e284415f5be3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=79e70c946e5440e39ee1d7cbeb0a4160)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:21:24.641810 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18a7fb0-f294-4939-b5c4-e284415f5be3 admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:21:24.754637 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-e18a7fb0-f294-4939-b5c4-e284415f5be3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '79e70c946e5440e39ee1d7cbeb0a4160', 'request_id': 'req-e18a7fb0-f294-4939-b5c4-e284415f5be3'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:21:24.761651 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-e18a7fb0-f294-4939-b5c4-e284415f5be3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '79e70c946e5440e39ee1d7cbeb0a4160', 'request_id': 'req-e18a7fb0-f294-4939-b5c4-e284415f5be3'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:21:24.845675 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-e18a7fb0-f294-4939-b5c4-e284415f5be3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '79e70c946e5440e39ee1d7cbeb0a4160', 'request_id': 'req-e18a7fb0-f294-4939-b5c4-e284415f5be3'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:21:24.857135 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-e18a7fb0-f294-4939-b5c4-e284415f5be3 admin admin] The token cache is being invalidate because project 79e70c946e5440e39ee1d7cbeb0a4160 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80539) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 12:21:24.857419 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-e18a7fb0-f294-4939-b5c4-e284415f5be3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:21:24.858364 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-e18a7fb0-f294-4939-b5c4-e284415f5be3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:21:24.859973 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1116/2252] 10.222.0.23 () {58 vars in 1232 bytes} [Thu May 19 12:21:24 2022] DELETE /identity/v3/projects/79e70c946e5440e39ee1d7cbeb0a4160 => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:21:24.866525 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5ef57c-00a6-45d1-9982-d9091f8a3ea5 None None] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:21:24.867116 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5ef57c-00a6-45d1-9982-d9091f8a3ea5 None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:21:24.867116 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5ef57c-00a6-45d1-9982-d9091f8a3ea5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:21:24.886858 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.auth.core [None req-8f5ef57c-00a6-45d1-9982-d9091f8a3ea5 None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:21:24.888010 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-8f5ef57c-00a6-45d1-9982-d9091f8a3ea5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:21:24.908209 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:21:24.951158 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1137/2253] 10.222.0.23 () {58 vars in 957 bytes} [Thu May 19 12:21:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:21:24.958852 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a841e356-85b6-46d4-87f3-1a213c993816 None None] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:21:24.959143 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a841e356-85b6-46d4-87f3-1a213c993816 None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:21:24.959427 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a841e356-85b6-46d4-87f3-1a213c993816 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:21:24.978677 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.auth.core [None req-a841e356-85b6-46d4-87f3-1a213c993816 None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:21:24.979935 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-a841e356-85b6-46d4-87f3-1a213c993816 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:21:24.988296 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1117/2254] 10.222.0.23 () {58 vars in 957 bytes} [Thu May 19 12:21:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:21:24.993799 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5ef57c-00a6-45d1-9982-d9091f8a3ea5 None None] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:21:25.004370 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5ef57c-00a6-45d1-9982-d9091f8a3ea5 None None] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:21:25.009567 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459e8790-2152-42ef-a9fd-919ed4fea0f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:21:25.010257 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459e8790-2152-42ef-a9fd-919ed4fea0f2 admin admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:21:25.010588 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459e8790-2152-42ef-a9fd-919ed4fea0f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:21:25.010870 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459e8790-2152-42ef-a9fd-919ed4fea0f2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:21:25.011607 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459e8790-2152-42ef-a9fd-919ed4fea0f2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:21:25.011968 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459e8790-2152-42ef-a9fd-919ed4fea0f2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:21:25.013588 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459e8790-2152-42ef-a9fd-919ed4fea0f2 admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:21:25.026391 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1138/2255] 10.222.0.23 () {58 vars in 1152 bytes} [Thu May 19 12:21:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:21:25.030883 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a841e356-85b6-46d4-87f3-1a213c993816 None None] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:21:25.040903 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a841e356-85b6-46d4-87f3-1a213c993816 None None] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:21:25.045779 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3c0391-0354-4fc7-9f45-00c4fca4453c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:21:25.046445 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3c0391-0354-4fc7-9f45-00c4fca4453c admin admin] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:21:25.046732 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3c0391-0354-4fc7-9f45-00c4fca4453c admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:21:25.047016 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3c0391-0354-4fc7-9f45-00c4fca4453c admin admin] PATH_INFO: `/v3/projects` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:21:25.047725 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3c0391-0354-4fc7-9f45-00c4fca4453c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:21:25.049704 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3c0391-0354-4fc7-9f45-00c4fca4453c admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:21:25.060293 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:21:25.224917 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1118/2256] 10.222.0.23 () {60 vars in 1152 bytes} [Thu May 19 12:21:25 2022] POST /identity/v3/projects => generated 383 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 12:21:25.229764 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459e8790-2152-42ef-a9fd-919ed4fea0f2 admin admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:21:25.240558 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459e8790-2152-42ef-a9fd-919ed4fea0f2 admin admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:21:25.299321 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a19098e-239c-4bcb-b98d-3e6036ac4767 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:21:25.300026 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a19098e-239c-4bcb-b98d-3e6036ac4767 admin admin] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:21:25.300329 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a19098e-239c-4bcb-b98d-3e6036ac4767 admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:21:25.300588 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a19098e-239c-4bcb-b98d-3e6036ac4767 admin admin] PATH_INFO: `/v3/users` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:21:25.301370 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a19098e-239c-4bcb-b98d-3e6036ac4767 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:21:25.303316 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a19098e-239c-4bcb-b98d-3e6036ac4767 admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:21:25.441071 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1139/2257] 10.222.0.23 () {60 vars in 1143 bytes} [Thu May 19 12:21:25 2022] POST /identity/v3/users => generated 292 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 12:21:25.445855 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3c0391-0354-4fc7-9f45-00c4fca4453c admin admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:21:25.457251 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3c0391-0354-4fc7-9f45-00c4fca4453c admin admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:21:25.462614 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfc6e61-f1a8-4d7c-9da9-6f57b0f768e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:21:25.463273 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfc6e61-f1a8-4d7c-9da9-6f57b0f768e3 admin admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:21:25.463658 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfc6e61-f1a8-4d7c-9da9-6f57b0f768e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:21:25.464019 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfc6e61-f1a8-4d7c-9da9-6f57b0f768e3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:21:25.464751 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abfc6e61-f1a8-4d7c-9da9-6f57b0f768e3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:21:25.466369 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abfc6e61-f1a8-4d7c-9da9-6f57b0f768e3 admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:21:25.477145 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1119/2258] 10.222.0.23 () {58 vars in 1121 bytes} [Thu May 19 12:21:25 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 12:21:25.486296 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a19098e-239c-4bcb-b98d-3e6036ac4767 admin admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:21:25.497056 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a19098e-239c-4bcb-b98d-3e6036ac4767 admin admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:21:25.501682 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccf50c3-bfc4-45aa-8124-29ef06d36d18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:21:25.502347 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccf50c3-bfc4-45aa-8124-29ef06d36d18 admin admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:21:25.502640 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccf50c3-bfc4-45aa-8124-29ef06d36d18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:21:25.502924 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccf50c3-bfc4-45aa-8124-29ef06d36d18 admin admin] PATH_INFO: `/v3/roles` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:21:25.503660 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ccf50c3-bfc4-45aa-8124-29ef06d36d18 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:21:25.505317 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ccf50c3-bfc4-45aa-8124-29ef06d36d18 admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:21:25.516267 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1140/2259] 10.222.0.23 () {58 vars in 1121 bytes} [Thu May 19 12:21:25 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 12:21:25.520578 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfc6e61-f1a8-4d7c-9da9-6f57b0f768e3 admin admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:21:25.531280 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfc6e61-f1a8-4d7c-9da9-6f57b0f768e3 admin admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:21:25.535930 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f03b175-6a38-4de5-beb1-ad5910824cb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:21:25.536827 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f03b175-6a38-4de5-beb1-ad5910824cb6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:21:25.537112 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f03b175-6a38-4de5-beb1-ad5910824cb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:21:25.537382 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f03b175-6a38-4de5-beb1-ad5910824cb6 admin admin] PATH_INFO: `/v3/projects/37c1366ee71c446b9ec6fa532391b789/users/d53ee7f4a9b74ce1868715ed0177bf2a/roles/259eb400cdb4439db72d14456ae9cd08` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:21:25.552790 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f03b175-6a38-4de5-beb1-ad5910824cb6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=37c1366ee71c446b9ec6fa532391b789, user_id=d53ee7f4a9b74ce1868715ed0177bf2a, role_id=259eb400cdb4439db72d14456ae9cd08)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:21:25.555448 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f03b175-6a38-4de5-beb1-ad5910824cb6 admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:21:25.629035 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1120/2260] 10.222.0.23 () {60 vars in 1482 bytes} [Thu May 19 12:21:25 2022] PUT /identity/v3/projects/37c1366ee71c446b9ec6fa532391b789/users/d53ee7f4a9b74ce1868715ed0177bf2a/roles/259eb400cdb4439db72d14456ae9cd08 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:21:25.634845 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f1c424-ccfb-4436-9169-166fb505ce6c None None] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:21:25.635149 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f1c424-ccfb-4436-9169-166fb505ce6c None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:21:25.635434 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f1c424-ccfb-4436-9169-166fb505ce6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:21:25.655263 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.auth.core [None req-c5f1c424-ccfb-4436-9169-166fb505ce6c None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:21:25.656532 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-c5f1c424-ccfb-4436-9169-166fb505ce6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:21:25.717059 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1141/2261] 10.222.0.23 () {58 vars in 957 bytes} [Thu May 19 12:21:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:21:25.726722 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f03b175-6a38-4de5-beb1-ad5910824cb6 admin admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:21:25.728509 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-8f03b175-6a38-4de5-beb1-ad5910824cb6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:21:25.788367 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f03b175-6a38-4de5-beb1-ad5910824cb6 admin admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:21:25.792139 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234967a6-7143-4c28-ac32-66ba476d62ab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a00b3fa7497456895e5e0bf105c4fc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd70d25bf69314820b8aaecb65576978a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:21:25.792776 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234967a6-7143-4c28-ac32-66ba476d62ab service neutron] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:21:25.793063 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234967a6-7143-4c28-ac32-66ba476d62ab service neutron] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:21:25.793337 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234967a6-7143-4c28-ac32-66ba476d62ab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:21:25.802679 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234967a6-7143-4c28-ac32-66ba476d62ab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:21:25.804956 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234967a6-7143-4c28-ac32-66ba476d62ab service neutron] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:21:25.815005 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1121/2262] 10.222.0.23 () {60 vars in 1365 bytes} [Thu May 19 12:21:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 12:21:32.847704 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72c0c6f-fe72-4de5-847e-b858aad083d8 None None] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:21:32.848277 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72c0c6f-fe72-4de5-847e-b858aad083d8 None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:21:32.848453 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72c0c6f-fe72-4de5-847e-b858aad083d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:21:32.880302 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.auth.core [None req-e72c0c6f-fe72-4de5-847e-b858aad083d8 None None] MFA Rules not processed for user `d53ee7f4a9b74ce1868715ed0177bf2a`. Rule list: `[]` (Enabled: `True`). {{(pid=80540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:21:32.885880 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-e72c0c6f-fe72-4de5-847e-b858aad083d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:21:32.910372 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:21:32.960593 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1142/2263] 10.222.0.23 () {58 vars in 957 bytes} [Thu May 19 12:21:32 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:21:32.965119 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234967a6-7143-4c28-ac32-66ba476d62ab service neutron] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:21:32.974986 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234967a6-7143-4c28-ac32-66ba476d62ab service neutron] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:21:32.980846 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09a5891-f6e0-4b9c-ba38-488c4a74e37e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:21:32.981560 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09a5891-f6e0-4b9c-ba38-488c4a74e37e admin admin] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:21:32.981862 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09a5891-f6e0-4b9c-ba38-488c4a74e37e admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:21:32.982138 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09a5891-f6e0-4b9c-ba38-488c4a74e37e admin admin] PATH_INFO: `/v3/projects` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:21:32.982835 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c09a5891-f6e0-4b9c-ba38-488c4a74e37e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:21:32.985121 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c09a5891-f6e0-4b9c-ba38-488c4a74e37e admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:21:33.157829 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1122/2264] 10.222.0.23 () {60 vars in 1152 bytes} [Thu May 19 12:21:32 2022] POST /identity/v3/projects => generated 385 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 12:21:33.161365 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72c0c6f-fe72-4de5-847e-b858aad083d8 None None] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:21:33.176958 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72c0c6f-fe72-4de5-847e-b858aad083d8 None None] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:21:33.239214 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c9f482-7cca-40df-ae33-93bf8a013f58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:21:33.239843 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c9f482-7cca-40df-ae33-93bf8a013f58 admin admin] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:21:33.240137 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c9f482-7cca-40df-ae33-93bf8a013f58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:21:33.240445 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c9f482-7cca-40df-ae33-93bf8a013f58 admin admin] PATH_INFO: `/v3/users` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:21:33.241186 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c9f482-7cca-40df-ae33-93bf8a013f58 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:21:33.243156 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c9f482-7cca-40df-ae33-93bf8a013f58 admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:21:33.572833 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1143/2265] 10.222.0.23 () {60 vars in 1143 bytes} [Thu May 19 12:21:33 2022] POST /identity/v3/users => generated 299 bytes in 412 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 12:21:33.587464 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09a5891-f6e0-4b9c-ba38-488c4a74e37e admin admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:21:33.595187 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09a5891-f6e0-4b9c-ba38-488c4a74e37e admin admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:21:33.609258 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea77d45-6e81-4857-ac97-57fcc5df7748 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:21:33.609860 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea77d45-6e81-4857-ac97-57fcc5df7748 admin admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:21:33.610149 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea77d45-6e81-4857-ac97-57fcc5df7748 admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:21:33.610451 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea77d45-6e81-4857-ac97-57fcc5df7748 admin admin] PATH_INFO: `/v3/roles` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:21:33.611534 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea77d45-6e81-4857-ac97-57fcc5df7748 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:21:33.613422 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea77d45-6e81-4857-ac97-57fcc5df7748 admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:21:33.626824 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1123/2266] 10.222.0.23 () {58 vars in 1121 bytes} [Thu May 19 12:21:33 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 19 12:21:33.631025 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c9f482-7cca-40df-ae33-93bf8a013f58 admin admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:21:33.642510 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c9f482-7cca-40df-ae33-93bf8a013f58 admin admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:21:33.647523 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e19385f-0c5a-49dd-a70a-b8e1a46ab9fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:21:33.648580 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e19385f-0c5a-49dd-a70a-b8e1a46ab9fa admin admin] REQUEST_METHOD: `PUT` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:21:33.648927 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e19385f-0c5a-49dd-a70a-b8e1a46ab9fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:21:33.649297 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e19385f-0c5a-49dd-a70a-b8e1a46ab9fa admin admin] PATH_INFO: `/v3/projects/22ac51f7867c4e50a7ee4863f7f7aa28/users/ef2523c3639448cf8697032a07e79fb4/roles/9db5671c65d3414ea2682afa2ccf1403` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:21:33.672162 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e19385f-0c5a-49dd-a70a-b8e1a46ab9fa admin admin] RBAC: Authorizing `identity:create_grant(project_id=22ac51f7867c4e50a7ee4863f7f7aa28, user_id=ef2523c3639448cf8697032a07e79fb4, role_id=9db5671c65d3414ea2682afa2ccf1403)` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:21:33.672162 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e19385f-0c5a-49dd-a70a-b8e1a46ab9fa admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:21:33.930660 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1144/2267] 10.222.0.23 () {60 vars in 1482 bytes} [Thu May 19 12:21:33 2022] PUT /identity/v3/projects/22ac51f7867c4e50a7ee4863f7f7aa28/users/ef2523c3639448cf8697032a07e79fb4/roles/9db5671c65d3414ea2682afa2ccf1403 => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:21:33.933141 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea77d45-6e81-4857-ac97-57fcc5df7748 admin admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:21:33.944090 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea77d45-6e81-4857-ac97-57fcc5df7748 admin admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:21:34.009179 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528e16eb-6fbb-4129-a1a7-e4e360ebc8c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:21:34.010014 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528e16eb-6fbb-4129-a1a7-e4e360ebc8c3 admin admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:21:34.010302 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528e16eb-6fbb-4129-a1a7-e4e360ebc8c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:21:34.010578 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528e16eb-6fbb-4129-a1a7-e4e360ebc8c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:21:34.011334 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528e16eb-6fbb-4129-a1a7-e4e360ebc8c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:21:34.013143 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528e16eb-6fbb-4129-a1a7-e4e360ebc8c3 admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:21:34.037944 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1124/2268] 10.222.0.23 () {58 vars in 1121 bytes} [Thu May 19 12:21:33 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 12:21:34.039882 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e19385f-0c5a-49dd-a70a-b8e1a46ab9fa admin admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:21:34.050676 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e19385f-0c5a-49dd-a70a-b8e1a46ab9fa admin admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:21:34.056284 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d2b4ab-d318-4827-b170-7e66f5c51efe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:21:34.057186 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d2b4ab-d318-4827-b170-7e66f5c51efe admin admin] REQUEST_METHOD: `PUT` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:21:34.057471 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d2b4ab-d318-4827-b170-7e66f5c51efe admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:21:34.057748 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d2b4ab-d318-4827-b170-7e66f5c51efe admin admin] PATH_INFO: `/v3/projects/22ac51f7867c4e50a7ee4863f7f7aa28/users/ef2523c3639448cf8697032a07e79fb4/roles/9db5671c65d3414ea2682afa2ccf1403` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:21:34.059745 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d2b4ab-d318-4827-b170-7e66f5c51efe admin admin] RBAC: Authorizing `identity:create_grant(project_id=22ac51f7867c4e50a7ee4863f7f7aa28, user_id=ef2523c3639448cf8697032a07e79fb4, role_id=9db5671c65d3414ea2682afa2ccf1403)` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:21:34.062527 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d2b4ab-d318-4827-b170-7e66f5c51efe admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:21:34.076720 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1145/2269] 10.222.0.23 () {60 vars in 1482 bytes} [Thu May 19 12:21:34 2022] PUT /identity/v3/projects/22ac51f7867c4e50a7ee4863f7f7aa28/users/ef2523c3639448cf8697032a07e79fb4/roles/9db5671c65d3414ea2682afa2ccf1403 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:21:41.028963 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528e16eb-6fbb-4129-a1a7-e4e360ebc8c3 admin admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:21:41.031467 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-528e16eb-6fbb-4129-a1a7-e4e360ebc8c3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:21:41.074134 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:21:41.108486 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b263988-5c9e-45eb-a9c8-f180dd2e6ee0 None None] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:21:41.109084 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b263988-5c9e-45eb-a9c8-f180dd2e6ee0 None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:21:41.109299 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b263988-5c9e-45eb-a9c8-f180dd2e6ee0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:21:41.148016 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.auth.core [None req-1b263988-5c9e-45eb-a9c8-f180dd2e6ee0 None None] MFA Rules not processed for user `ef2523c3639448cf8697032a07e79fb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:21:41.149646 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-1b263988-5c9e-45eb-a9c8-f180dd2e6ee0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:21:41.164686 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528e16eb-6fbb-4129-a1a7-e4e360ebc8c3 admin admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:21:41.231418 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:21:41.238197 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1146/2270] 10.222.0.23 () {58 vars in 957 bytes} [Thu May 19 12:21:41 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:21:41.251165 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d28660a-be47-4cc6-b7f3-ef9c76794496 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50f937eed4f3414fa6f19cab2a84eb5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd70d25bf69314820b8aaecb65576978a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:21:41.251165 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d28660a-be47-4cc6-b7f3-ef9c76794496 service placement] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:21:41.251165 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d28660a-be47-4cc6-b7f3-ef9c76794496 service placement] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:21:41.251165 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d28660a-be47-4cc6-b7f3-ef9c76794496 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:21:41.251479 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b263988-5c9e-45eb-a9c8-f180dd2e6ee0 None None] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:21:41.252508 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-0d28660a-be47-4cc6-b7f3-ef9c76794496 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:21:41.252595 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-1b263988-5c9e-45eb-a9c8-f180dd2e6ee0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:21:41.265197 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d28660a-be47-4cc6-b7f3-ef9c76794496 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:21:41.266856 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d28660a-be47-4cc6-b7f3-ef9c76794496 service placement] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:21:41.278625 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1125/2271] 10.222.0.23 () {60 vars in 1365 bytes} [Thu May 19 12:21:41 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 250 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:21:41.341045 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b263988-5c9e-45eb-a9c8-f180dd2e6ee0 None None] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:21:41.346218 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7405f3-5288-4b2a-bfb6-911b2f3efa9d service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '21697802f2be42f9baa8e07490d5ebdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd70d25bf69314820b8aaecb65576978a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:21:41.346931 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7405f3-5288-4b2a-bfb6-911b2f3efa9d service glance] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:21:41.347216 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7405f3-5288-4b2a-bfb6-911b2f3efa9d service glance] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:21:41.347692 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7405f3-5288-4b2a-bfb6-911b2f3efa9d service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:21:41.415982 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7405f3-5288-4b2a-bfb6-911b2f3efa9d service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:21:41.417447 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7405f3-5288-4b2a-bfb6-911b2f3efa9d service glance] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:21:41.428136 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1147/2272] 10.222.0.23 () {60 vars in 1365 bytes} [Thu May 19 12:21:41 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 12:21:41.738232 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f38e12-7aa7-49bd-aa10-aecb92473345 None None] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:21:41.738527 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f38e12-7aa7-49bd-aa10-aecb92473345 None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:21:41.738833 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f38e12-7aa7-49bd-aa10-aecb92473345 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:21:41.756563 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.auth.core [None req-35f38e12-7aa7-49bd-aa10-aecb92473345 None None] MFA Rules not processed for user `0c3181fb4f584a549a786adfefdf3dfb`. Rule list: `[]` (Enabled: `True`). {{(pid=80539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:21:41.757872 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-35f38e12-7aa7-49bd-aa10-aecb92473345 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:21:41.817539 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1126/2273] 10.222.0.23 () {58 vars in 1009 bytes} [Thu May 19 12:21:41 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:21:41.830880 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7405f3-5288-4b2a-bfb6-911b2f3efa9d service glance] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:21:41.833138 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-4c7405f3-5288-4b2a-bfb6-911b2f3efa9d service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:21:41.908177 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7405f3-5288-4b2a-bfb6-911b2f3efa9d service glance] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:21:41.913237 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782623fd-f211-45a8-ace6-0ef8b7740a9d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15cbfc73ae1243d2a8b1e70f9cd99876', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd70d25bf69314820b8aaecb65576978a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:21:41.914497 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782623fd-f211-45a8-ace6-0ef8b7740a9d service swift] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:21:41.914825 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782623fd-f211-45a8-ace6-0ef8b7740a9d service swift] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:21:41.915082 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782623fd-f211-45a8-ace6-0ef8b7740a9d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:21:41.923756 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782623fd-f211-45a8-ace6-0ef8b7740a9d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:21:41.925462 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782623fd-f211-45a8-ace6-0ef8b7740a9d service swift] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:21:41.935189 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1148/2274] 10.222.0.23 () {60 vars in 1384 bytes} [Thu May 19 12:21:41 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 19 12:21:53.126595 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=82 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:21:53.214342 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=90 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:21:55.622892 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd9ec22-2138-4c11-9282-b87cd5910397 None None] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:21:55.623570 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd9ec22-2138-4c11-9282-b87cd5910397 None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:21:55.623570 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd9ec22-2138-4c11-9282-b87cd5910397 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:21:55.642682 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.auth.core [None req-0cd9ec22-2138-4c11-9282-b87cd5910397 None None] MFA Rules not processed for user `d53ee7f4a9b74ce1868715ed0177bf2a`. Rule list: `[]` (Enabled: `True`). {{(pid=80539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:21:55.643920 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-0cd9ec22-2138-4c11-9282-b87cd5910397 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:21:55.652146 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1127/2275] 10.222.0.23 () {58 vars in 957 bytes} [Thu May 19 12:21:55 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:22:00.812036 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db874d8-a6f5-49a4-89dc-55171fd69b50 None None] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:22:00.813399 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db874d8-a6f5-49a4-89dc-55171fd69b50 None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:22:00.814008 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db874d8-a6f5-49a4-89dc-55171fd69b50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:22:00.840325 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.auth.core [None req-1db874d8-a6f5-49a4-89dc-55171fd69b50 None None] MFA Rules not processed for user `0c3181fb4f584a549a786adfefdf3dfb`. Rule list: `[]` (Enabled: `True`). {{(pid=80540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:22:00.841962 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-1db874d8-a6f5-49a4-89dc-55171fd69b50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:22:00.862903 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1149/2276] 10.222.0.23 () {58 vars in 1009 bytes} [Thu May 19 12:22:00 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:22:00.877677 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd9ec22-2138-4c11-9282-b87cd5910397 None None] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:22:00.896324 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd9ec22-2138-4c11-9282-b87cd5910397 None None] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:22:00.905746 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc194fc6-533d-4db8-be4a-416268dd2c6b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15cbfc73ae1243d2a8b1e70f9cd99876', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd70d25bf69314820b8aaecb65576978a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:22:00.907976 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc194fc6-533d-4db8-be4a-416268dd2c6b service swift] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:22:00.907976 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc194fc6-533d-4db8-be4a-416268dd2c6b service swift] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:22:00.907976 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc194fc6-533d-4db8-be4a-416268dd2c6b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:22:00.924796 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc194fc6-533d-4db8-be4a-416268dd2c6b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:22:00.926706 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc194fc6-533d-4db8-be4a-416268dd2c6b service swift] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:22:00.938228 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1128/2277] 10.222.0.23 () {60 vars in 1384 bytes} [Thu May 19 12:22:00 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 19 12:22:04.600302 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33f4007-4161-4fcb-a155-a530651b5424 None None] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:22:04.600302 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33f4007-4161-4fcb-a155-a530651b5424 None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:22:04.600302 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33f4007-4161-4fcb-a155-a530651b5424 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:22:04.618441 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.auth.core [None req-f33f4007-4161-4fcb-a155-a530651b5424 None None] MFA Rules not processed for user `dd39ddcf4ab045b98a737ee529a2dcce`. Rule list: `[]` (Enabled: `True`). {{(pid=80540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:22:04.619664 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-f33f4007-4161-4fcb-a155-a530651b5424 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:22:04.685882 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1150/2278] 10.222.0.23 () {58 vars in 957 bytes} [Thu May 19 12:22:04 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:22:10.934319 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=9 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:22:14.677798 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=28 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:22:18.766513 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc194fc6-533d-4db8-be4a-416268dd2c6b service swift] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:22:18.842626 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc194fc6-533d-4db8-be4a-416268dd2c6b service swift] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:22:18.843176 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc61de4-d9ee-45a1-8e78-a7f29274e85f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a00b3fa7497456895e5e0bf105c4fc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd70d25bf69314820b8aaecb65576978a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:22:18.843958 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc61de4-d9ee-45a1-8e78-a7f29274e85f service neutron] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:22:18.844317 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc61de4-d9ee-45a1-8e78-a7f29274e85f service neutron] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:22:18.844665 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc61de4-d9ee-45a1-8e78-a7f29274e85f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:22:18.846814 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-6dc61de4-d9ee-45a1-8e78-a7f29274e85f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:22:18.926288 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc61de4-d9ee-45a1-8e78-a7f29274e85f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:22:18.926288 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc61de4-d9ee-45a1-8e78-a7f29274e85f service neutron] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:22:18.934329 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1129/2279] 10.222.0.23 () {60 vars in 1365 bytes} [Thu May 19 12:22:18 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:22:28.927406 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=50 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:22:46.640416 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33f4007-4161-4fcb-a155-a530651b5424 None None] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:22:46.651202 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33f4007-4161-4fcb-a155-a530651b5424 None None] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:22:46.655038 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e637d27d-14ca-48b7-9557-5d66dfdf6ab3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a00b3fa7497456895e5e0bf105c4fc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd70d25bf69314820b8aaecb65576978a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:22:46.656489 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e637d27d-14ca-48b7-9557-5d66dfdf6ab3 service neutron] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:22:46.656489 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e637d27d-14ca-48b7-9557-5d66dfdf6ab3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:22:46.656489 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e637d27d-14ca-48b7-9557-5d66dfdf6ab3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:22:46.666113 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e637d27d-14ca-48b7-9557-5d66dfdf6ab3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:22:46.667637 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e637d27d-14ca-48b7-9557-5d66dfdf6ab3 service neutron] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:22:46.677799 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1151/2280] 10.222.0.23 () {60 vars in 1365 bytes} [Thu May 19 12:22:46 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:22:51.425908 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc61de4-d9ee-45a1-8e78-a7f29274e85f service neutron] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:22:51.427405 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-6dc61de4-d9ee-45a1-8e78-a7f29274e85f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:22:51.440591 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc61de4-d9ee-45a1-8e78-a7f29274e85f service neutron] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:22:51.445063 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167f8dbe-438c-46fd-ae49-9e31776e813f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a00b3fa7497456895e5e0bf105c4fc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd70d25bf69314820b8aaecb65576978a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:22:51.445724 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167f8dbe-438c-46fd-ae49-9e31776e813f service neutron] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:22:51.446010 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167f8dbe-438c-46fd-ae49-9e31776e813f service neutron] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:22:51.449072 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167f8dbe-438c-46fd-ae49-9e31776e813f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:22:51.449072 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-167f8dbe-438c-46fd-ae49-9e31776e813f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:22:51.531982 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167f8dbe-438c-46fd-ae49-9e31776e813f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:22:51.533724 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167f8dbe-438c-46fd-ae49-9e31776e813f service neutron] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:22:51.547607 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1130/2281] 10.222.0.23 () {60 vars in 1365 bytes} [Thu May 19 12:22:51 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:22:52.596156 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e637d27d-14ca-48b7-9557-5d66dfdf6ab3 service neutron] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:22:52.605157 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e637d27d-14ca-48b7-9557-5d66dfdf6ab3 service neutron] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:22:52.608813 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e613c2-75f7-47b8-8df7-e3e96054565e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a00b3fa7497456895e5e0bf105c4fc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd70d25bf69314820b8aaecb65576978a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:22:52.609709 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e613c2-75f7-47b8-8df7-e3e96054565e service neutron] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:22:52.609999 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e613c2-75f7-47b8-8df7-e3e96054565e service neutron] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:22:52.610278 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e613c2-75f7-47b8-8df7-e3e96054565e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:22:52.612170 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-02e613c2-75f7-47b8-8df7-e3e96054565e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:22:52.625814 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e613c2-75f7-47b8-8df7-e3e96054565e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:22:52.627249 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e613c2-75f7-47b8-8df7-e3e96054565e service neutron] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:22:52.638658 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1152/2282] 10.222.0.23 () {60 vars in 1365 bytes} [Thu May 19 12:22:52 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:23:00.564343 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167f8dbe-438c-46fd-ae49-9e31776e813f service neutron] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:23:00.566509 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-167f8dbe-438c-46fd-ae49-9e31776e813f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:23:00.582409 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167f8dbe-438c-46fd-ae49-9e31776e813f service neutron] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:23:00.589246 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a7d445-514a-43b9-84c8-b0f82c714634 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34a98d75d84b4c67981377e0f26eca62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd70d25bf69314820b8aaecb65576978a', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:23:00.590048 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a7d445-514a-43b9-84c8-b0f82c714634 service nova] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:23:00.590400 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a7d445-514a-43b9-84c8-b0f82c714634 service nova] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:23:00.590753 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a7d445-514a-43b9-84c8-b0f82c714634 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:23:00.593020 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-25a7d445-514a-43b9-84c8-b0f82c714634 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:23:00.656772 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a7d445-514a-43b9-84c8-b0f82c714634 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:23:00.658416 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a7d445-514a-43b9-84c8-b0f82c714634 service nova] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:23:00.668198 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1131/2283] 10.222.0.23 () {60 vars in 1365 bytes} [Thu May 19 12:23:00 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:23:02.633782 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=12 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:23:04.881510 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc658c6-6d7a-4d48-9dfc-92616b0d0716 None None] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:23:04.882062 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc658c6-6d7a-4d48-9dfc-92616b0d0716 None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:23:04.887730 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc658c6-6d7a-4d48-9dfc-92616b0d0716 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:23:04.901769 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.auth.core [None req-8bc658c6-6d7a-4d48-9dfc-92616b0d0716 None None] MFA Rules not processed for user `0c3181fb4f584a549a786adfefdf3dfb`. Rule list: `[]` (Enabled: `True`). {{(pid=80540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:23:04.902985 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-8bc658c6-6d7a-4d48-9dfc-92616b0d0716 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:23:04.910259 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1153/2284] 10.222.0.23 () {58 vars in 1009 bytes} [Thu May 19 12:23:04 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:23:04.952278 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a7d445-514a-43b9-84c8-b0f82c714634 service nova] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:23:04.964655 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a7d445-514a-43b9-84c8-b0f82c714634 service nova] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:23:04.969988 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e23997d-6a29-4376-9b30-f6d0c91fbdcd service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15cbfc73ae1243d2a8b1e70f9cd99876', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd70d25bf69314820b8aaecb65576978a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:23:04.970662 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e23997d-6a29-4376-9b30-f6d0c91fbdcd service swift] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:23:04.970948 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e23997d-6a29-4376-9b30-f6d0c91fbdcd service swift] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:23:04.971225 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e23997d-6a29-4376-9b30-f6d0c91fbdcd service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:23:04.980655 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e23997d-6a29-4376-9b30-f6d0c91fbdcd service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:23:04.982240 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e23997d-6a29-4376-9b30-f6d0c91fbdcd service swift] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:23:04.991761 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1132/2285] 10.222.0.23 () {60 vars in 1384 bytes} [Thu May 19 12:23:04 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) May 19 12:23:14.894544 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=3 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:23:14.988012 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=68 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:23:18.207121 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc658c6-6d7a-4d48-9dfc-92616b0d0716 None None] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:23:18.217604 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc658c6-6d7a-4d48-9dfc-92616b0d0716 None None] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:23:18.221452 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793bbc05-e5d4-4a7e-b778-0e8a25991f4e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a00b3fa7497456895e5e0bf105c4fc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd70d25bf69314820b8aaecb65576978a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:23:18.222121 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793bbc05-e5d4-4a7e-b778-0e8a25991f4e service neutron] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:23:18.222412 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793bbc05-e5d4-4a7e-b778-0e8a25991f4e service neutron] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:23:18.222692 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793bbc05-e5d4-4a7e-b778-0e8a25991f4e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:23:18.224555 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-793bbc05-e5d4-4a7e-b778-0e8a25991f4e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:23:18.235101 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-793bbc05-e5d4-4a7e-b778-0e8a25991f4e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:23:18.236616 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-793bbc05-e5d4-4a7e-b778-0e8a25991f4e service neutron] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:23:18.245035 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1154/2286] 10.222.0.23 () {60 vars in 1365 bytes} [Thu May 19 12:23:18 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:23:21.076498 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fddf27-b656-4cd8-920b-8122db73ac83 None None] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:23:21.077027 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fddf27-b656-4cd8-920b-8122db73ac83 None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:23:21.077027 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fddf27-b656-4cd8-920b-8122db73ac83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:23:21.095068 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.auth.core [None req-93fddf27-b656-4cd8-920b-8122db73ac83 None None] MFA Rules not processed for user `8e445569dc5641ed962b0ac977850a6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:23:21.096296 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-93fddf27-b656-4cd8-920b-8122db73ac83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:23:21.103431 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1133/2287] 10.222.0.23 () {58 vars in 957 bytes} [Thu May 19 12:23:21 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:23:26.800959 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b281c4ef-db21-45bb-9deb-493ed81d4563 None None] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:23:26.801685 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b281c4ef-db21-45bb-9deb-493ed81d4563 None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:23:26.801685 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b281c4ef-db21-45bb-9deb-493ed81d4563 None None] PATH_INFO: `/` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:23:26.802905 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1155/2288] 10.222.0.23 () {54 vars in 921 bytes} [Thu May 19 12:23:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 12:23:26.809487 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37a8012-8b3a-4e6c-b82e-117e40d73eca None None] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:23:26.809487 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37a8012-8b3a-4e6c-b82e-117e40d73eca None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:23:26.810270 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37a8012-8b3a-4e6c-b82e-117e40d73eca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:23:26.850584 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.auth.core [None req-a37a8012-8b3a-4e6c-b82e-117e40d73eca None None] MFA Rules not processed for user `34a98d75d84b4c67981377e0f26eca62`. Rule list: `[]` (Enabled: `True`). {{(pid=80539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:23:26.852703 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-a37a8012-8b3a-4e6c-b82e-117e40d73eca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:23:26.862385 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1134/2289] 10.222.0.23 () {58 vars in 1018 bytes} [Thu May 19 12:23:26 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:23:26.874132 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b281c4ef-db21-45bb-9deb-493ed81d4563 None None] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:23:26.885921 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b281c4ef-db21-45bb-9deb-493ed81d4563 None None] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:23:26.889766 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520a8ae3-2387-4a31-9125-5179a5653cf1 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '21697802f2be42f9baa8e07490d5ebdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd70d25bf69314820b8aaecb65576978a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:23:26.890395 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520a8ae3-2387-4a31-9125-5179a5653cf1 service glance] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:23:26.890680 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520a8ae3-2387-4a31-9125-5179a5653cf1 service glance] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:23:26.891037 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520a8ae3-2387-4a31-9125-5179a5653cf1 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:23:26.901848 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520a8ae3-2387-4a31-9125-5179a5653cf1 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:23:26.903364 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520a8ae3-2387-4a31-9125-5179a5653cf1 service glance] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:23:26.914131 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1156/2290] 10.222.0.23 () {60 vars in 1365 bytes} [Thu May 19 12:23:26 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:23:27.212455 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a929116a-60dd-4ea2-9caa-557a0dbe86e8 None None] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:23:27.212672 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a929116a-60dd-4ea2-9caa-557a0dbe86e8 None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:23:27.212953 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a929116a-60dd-4ea2-9caa-557a0dbe86e8 None None] PATH_INFO: `/` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:23:27.214369 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1135/2291] 10.222.0.23 () {54 vars in 921 bytes} [Thu May 19 12:23:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 12:23:27.219416 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7accfd3b-da1d-46e0-82db-968e29cd594b None None] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:23:27.219743 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7accfd3b-da1d-46e0-82db-968e29cd594b None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:23:27.226058 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7accfd3b-da1d-46e0-82db-968e29cd594b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:23:27.239511 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.auth.core [None req-7accfd3b-da1d-46e0-82db-968e29cd594b None None] MFA Rules not processed for user `1a00b3fa7497456895e5e0bf105c4fc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:23:27.240775 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-7accfd3b-da1d-46e0-82db-968e29cd594b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:23:27.248170 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1157/2292] 10.222.0.23 () {58 vars in 1018 bytes} [Thu May 19 12:23:27 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:23:27.260773 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a929116a-60dd-4ea2-9caa-557a0dbe86e8 None None] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:23:27.271749 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a929116a-60dd-4ea2-9caa-557a0dbe86e8 None None] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:23:27.276242 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108ec4c9-a36c-498f-8613-8f9e95927e8d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a00b3fa7497456895e5e0bf105c4fc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd70d25bf69314820b8aaecb65576978a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:23:27.276897 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108ec4c9-a36c-498f-8613-8f9e95927e8d service neutron] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:23:27.277182 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108ec4c9-a36c-498f-8613-8f9e95927e8d service neutron] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:23:27.277455 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108ec4c9-a36c-498f-8613-8f9e95927e8d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:23:27.285279 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108ec4c9-a36c-498f-8613-8f9e95927e8d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:23:27.286730 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108ec4c9-a36c-498f-8613-8f9e95927e8d service neutron] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:23:27.296758 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1136/2293] 10.222.0.23 () {60 vars in 1365 bytes} [Thu May 19 12:23:27 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:23:27.389263 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd5f064-3937-4b57-9d78-6d00ca182015 None None] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:23:27.389555 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd5f064-3937-4b57-9d78-6d00ca182015 None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:23:27.389838 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd5f064-3937-4b57-9d78-6d00ca182015 None None] PATH_INFO: `/` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:23:27.391225 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1158/2294] 10.222.0.23 () {54 vars in 921 bytes} [Thu May 19 12:23:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 12:23:27.396765 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f199b4-20a4-4126-9e68-970ce64fbdf3 None None] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:23:27.397055 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f199b4-20a4-4126-9e68-970ce64fbdf3 None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:23:27.397978 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f199b4-20a4-4126-9e68-970ce64fbdf3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:23:27.418395 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.auth.core [None req-40f199b4-20a4-4126-9e68-970ce64fbdf3 None None] MFA Rules not processed for user `34a98d75d84b4c67981377e0f26eca62`. Rule list: `[]` (Enabled: `True`). {{(pid=80539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:23:27.419633 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-40f199b4-20a4-4126-9e68-970ce64fbdf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:23:27.429726 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1137/2295] 10.222.0.23 () {58 vars in 1018 bytes} [Thu May 19 12:23:27 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:23:27.440212 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd5f064-3937-4b57-9d78-6d00ca182015 None None] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:23:27.451815 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd5f064-3937-4b57-9d78-6d00ca182015 None None] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:23:27.455565 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9184188-8416-4307-a115-709ddba88f42 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a00b3fa7497456895e5e0bf105c4fc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd70d25bf69314820b8aaecb65576978a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:23:27.456246 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9184188-8416-4307-a115-709ddba88f42 service neutron] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:23:27.456556 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9184188-8416-4307-a115-709ddba88f42 service neutron] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:23:27.456835 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9184188-8416-4307-a115-709ddba88f42 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:23:27.469184 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9184188-8416-4307-a115-709ddba88f42 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:23:27.470639 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9184188-8416-4307-a115-709ddba88f42 service neutron] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:23:27.483273 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1159/2296] 10.222.0.23 () {60 vars in 1365 bytes} [Thu May 19 12:23:27 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:23:27.552723 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f199b4-20a4-4126-9e68-970ce64fbdf3 None None] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:23:27.565259 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f199b4-20a4-4126-9e68-970ce64fbdf3 None None] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:23:27.570692 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd376151-1eb9-4445-9491-aa6b87fe8f11 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50f937eed4f3414fa6f19cab2a84eb5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd70d25bf69314820b8aaecb65576978a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:23:27.571426 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd376151-1eb9-4445-9491-aa6b87fe8f11 service placement] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:23:27.571774 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd376151-1eb9-4445-9491-aa6b87fe8f11 service placement] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:23:27.572140 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd376151-1eb9-4445-9491-aa6b87fe8f11 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:23:27.573937 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-bd376151-1eb9-4445-9491-aa6b87fe8f11 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:23:27.587684 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd376151-1eb9-4445-9491-aa6b87fe8f11 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:23:27.589327 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd376151-1eb9-4445-9491-aa6b87fe8f11 service placement] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:23:27.601739 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1138/2297] 10.222.0.23 () {60 vars in 1365 bytes} [Thu May 19 12:23:27 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:23:29.090413 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9184188-8416-4307-a115-709ddba88f42 service neutron] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:23:29.103095 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9184188-8416-4307-a115-709ddba88f42 service neutron] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:23:29.110844 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40786ade-d811-4aad-9cdf-883dd9420d12 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34a98d75d84b4c67981377e0f26eca62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd70d25bf69314820b8aaecb65576978a', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:23:29.111527 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40786ade-d811-4aad-9cdf-883dd9420d12 service nova] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:23:29.111848 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40786ade-d811-4aad-9cdf-883dd9420d12 service nova] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:23:29.112324 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40786ade-d811-4aad-9cdf-883dd9420d12 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:23:29.114496 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-40786ade-d811-4aad-9cdf-883dd9420d12 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:23:29.129554 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40786ade-d811-4aad-9cdf-883dd9420d12 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:23:29.131816 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40786ade-d811-4aad-9cdf-883dd9420d12 service nova] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:23:29.143186 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1160/2298] 10.222.0.23 () {60 vars in 1365 bytes} [Thu May 19 12:23:29 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:23:32.642898 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6752d0cf-d89c-4e93-bdad-8883a42f74cc None None] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:23:32.643393 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6752d0cf-d89c-4e93-bdad-8883a42f74cc None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:23:32.643488 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6752d0cf-d89c-4e93-bdad-8883a42f74cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:23:32.661008 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.auth.core [None req-6752d0cf-d89c-4e93-bdad-8883a42f74cc None None] MFA Rules not processed for user `0c3181fb4f584a549a786adfefdf3dfb`. Rule list: `[]` (Enabled: `True`). {{(pid=80539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:23:32.662127 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-6752d0cf-d89c-4e93-bdad-8883a42f74cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:23:32.669582 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1139/2299] 10.222.0.23 () {58 vars in 1009 bytes} [Thu May 19 12:23:32 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:23:32.682096 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40786ade-d811-4aad-9cdf-883dd9420d12 service nova] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:23:32.693258 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40786ade-d811-4aad-9cdf-883dd9420d12 service nova] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:23:32.698825 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7601aa-c3ff-4e7f-9363-687759c6361c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15cbfc73ae1243d2a8b1e70f9cd99876', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd70d25bf69314820b8aaecb65576978a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:23:32.699464 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7601aa-c3ff-4e7f-9363-687759c6361c service swift] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:23:32.699750 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7601aa-c3ff-4e7f-9363-687759c6361c service swift] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:23:32.700057 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7601aa-c3ff-4e7f-9363-687759c6361c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:23:32.708638 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7601aa-c3ff-4e7f-9363-687759c6361c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:23:32.710348 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7601aa-c3ff-4e7f-9363-687759c6361c service swift] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:23:32.718856 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1161/2300] 10.222.0.23 () {60 vars in 1384 bytes} [Thu May 19 12:23:32 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 19 12:23:42.655759 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=30 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:23:42.715980 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=33 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:23:47.573829 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6752d0cf-d89c-4e93-bdad-8883a42f74cc None None] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:23:47.588291 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6752d0cf-d89c-4e93-bdad-8883a42f74cc None None] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:23:47.595070 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0f497a-fa32-45aa-9510-421e92977b17 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34a98d75d84b4c67981377e0f26eca62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd70d25bf69314820b8aaecb65576978a', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:23:47.595850 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0f497a-fa32-45aa-9510-421e92977b17 service nova] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:23:47.596244 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0f497a-fa32-45aa-9510-421e92977b17 service nova] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:23:47.596588 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0f497a-fa32-45aa-9510-421e92977b17 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:23:47.598984 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-1f0f497a-fa32-45aa-9510-421e92977b17 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:23:47.677414 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0f497a-fa32-45aa-9510-421e92977b17 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:23:47.679857 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0f497a-fa32-45aa-9510-421e92977b17 service nova] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:23:47.692161 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1140/2301] 10.222.0.23 () {60 vars in 1365 bytes} [Thu May 19 12:23:47 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:23:57.687847 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=28 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:24:03.847737 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7601aa-c3ff-4e7f-9363-687759c6361c service swift] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:24:03.861145 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7601aa-c3ff-4e7f-9363-687759c6361c service swift] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:24:03.868338 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4532e7f-62a4-4274-bc5b-064a0db77261 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34a98d75d84b4c67981377e0f26eca62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd70d25bf69314820b8aaecb65576978a', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:24:03.869014 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4532e7f-62a4-4274-bc5b-064a0db77261 service nova] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:24:03.869305 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4532e7f-62a4-4274-bc5b-064a0db77261 service nova] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:24:03.869590 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4532e7f-62a4-4274-bc5b-064a0db77261 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:24:03.871566 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-d4532e7f-62a4-4274-bc5b-064a0db77261 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:24:03.884943 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4532e7f-62a4-4274-bc5b-064a0db77261 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:24:03.886473 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4532e7f-62a4-4274-bc5b-064a0db77261 service nova] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:24:03.897805 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1162/2302] 10.222.0.23 () {60 vars in 1365 bytes} [Thu May 19 12:24:03 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:24:06.688461 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162b212b-5157-45d4-b387-f27fa280e3e9 None None] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:24:06.689214 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162b212b-5157-45d4-b387-f27fa280e3e9 None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:24:06.689401 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162b212b-5157-45d4-b387-f27fa280e3e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:24:06.710014 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.auth.core [None req-162b212b-5157-45d4-b387-f27fa280e3e9 None None] MFA Rules not processed for user `0c3181fb4f584a549a786adfefdf3dfb`. Rule list: `[]` (Enabled: `True`). {{(pid=80539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:24:06.711236 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-162b212b-5157-45d4-b387-f27fa280e3e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:24:06.719356 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1141/2303] 10.222.0.23 () {58 vars in 967 bytes} [Thu May 19 12:24:06 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:24:06.748614 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4532e7f-62a4-4274-bc5b-064a0db77261 service nova] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:24:06.761230 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4532e7f-62a4-4274-bc5b-064a0db77261 service nova] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:24:06.766290 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639dd9a6-958e-415e-a5d0-0226075181b8 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15cbfc73ae1243d2a8b1e70f9cd99876', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd70d25bf69314820b8aaecb65576978a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:24:06.766972 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639dd9a6-958e-415e-a5d0-0226075181b8 service swift] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:24:06.767252 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639dd9a6-958e-415e-a5d0-0226075181b8 service swift] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:24:06.767526 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639dd9a6-958e-415e-a5d0-0226075181b8 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:24:06.777176 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639dd9a6-958e-415e-a5d0-0226075181b8 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:24:06.778782 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639dd9a6-958e-415e-a5d0-0226075181b8 service swift] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:24:06.787340 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1163/2304] 10.222.0.23 () {60 vars in 1384 bytes} [Thu May 19 12:24:06 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 19 12:24:16.704007 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=3 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:24:16.783662 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=12 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:24:35.136523 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162b212b-5157-45d4-b387-f27fa280e3e9 None None] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:24:35.149653 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162b212b-5157-45d4-b387-f27fa280e3e9 None None] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:24:35.155066 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1e3cd6-a37f-45cb-97a7-33f977408a5c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34a98d75d84b4c67981377e0f26eca62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd70d25bf69314820b8aaecb65576978a', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:24:35.155703 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1e3cd6-a37f-45cb-97a7-33f977408a5c service nova] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:24:35.156030 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1e3cd6-a37f-45cb-97a7-33f977408a5c service nova] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:24:35.156309 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1e3cd6-a37f-45cb-97a7-33f977408a5c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:24:35.158087 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-ea1e3cd6-a37f-45cb-97a7-33f977408a5c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:24:35.169592 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1e3cd6-a37f-45cb-97a7-33f977408a5c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:24:35.171195 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1e3cd6-a37f-45cb-97a7-33f977408a5c service nova] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:24:35.181931 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1142/2305] 10.222.0.23 () {60 vars in 1365 bytes} [Thu May 19 12:24:35 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 12:24:39.528201 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639dd9a6-958e-415e-a5d0-0226075181b8 service swift] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:24:39.545503 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639dd9a6-958e-415e-a5d0-0226075181b8 service swift] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:24:39.553235 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e1340e-6e1c-4bdb-b2b2-4f93d3df8067 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34a98d75d84b4c67981377e0f26eca62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd70d25bf69314820b8aaecb65576978a', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:24:39.554143 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e1340e-6e1c-4bdb-b2b2-4f93d3df8067 service nova] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:24:39.554414 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e1340e-6e1c-4bdb-b2b2-4f93d3df8067 service nova] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:24:39.554700 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e1340e-6e1c-4bdb-b2b2-4f93d3df8067 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:24:39.557442 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-e6e1340e-6e1c-4bdb-b2b2-4f93d3df8067 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:24:39.583046 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e1340e-6e1c-4bdb-b2b2-4f93d3df8067 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:24:39.585086 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e1340e-6e1c-4bdb-b2b2-4f93d3df8067 service nova] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:24:39.597180 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1164/2306] 10.222.0.23 () {60 vars in 1365 bytes} [Thu May 19 12:24:39 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 70 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:24:45.176201 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=6 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:24:49.592210 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=6 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:24:56.663154 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5cec5b-b5ea-4bd4-b3eb-f1b6f1fdd9d1 None None] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:24:56.663666 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5cec5b-b5ea-4bd4-b3eb-f1b6f1fdd9d1 None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:24:56.663763 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5cec5b-b5ea-4bd4-b3eb-f1b6f1fdd9d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:24:56.682430 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.auth.core [None req-ea5cec5b-b5ea-4bd4-b3eb-f1b6f1fdd9d1 None None] MFA Rules not processed for user `dd39ddcf4ab045b98a737ee529a2dcce`. Rule list: `[]` (Enabled: `True`). {{(pid=80539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:24:56.683647 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-ea5cec5b-b5ea-4bd4-b3eb-f1b6f1fdd9d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:24:56.690546 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1143/2307] 10.222.0.23 () {58 vars in 957 bytes} [Thu May 19 12:24:56 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:24:57.012685 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e1340e-6e1c-4bdb-b2b2-4f93d3df8067 service nova] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:24:57.024424 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e1340e-6e1c-4bdb-b2b2-4f93d3df8067 service nova] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:24:57.028347 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0766685b-54f9-4ad5-86f2-4229974fa9c4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a00b3fa7497456895e5e0bf105c4fc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd70d25bf69314820b8aaecb65576978a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:24:57.029048 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0766685b-54f9-4ad5-86f2-4229974fa9c4 service neutron] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:24:57.029282 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0766685b-54f9-4ad5-86f2-4229974fa9c4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:24:57.029804 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0766685b-54f9-4ad5-86f2-4229974fa9c4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:24:57.031625 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-0766685b-54f9-4ad5-86f2-4229974fa9c4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:24:57.042445 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0766685b-54f9-4ad5-86f2-4229974fa9c4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:24:57.043928 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0766685b-54f9-4ad5-86f2-4229974fa9c4 service neutron] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:24:57.052513 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1165/2308] 10.222.0.23 () {60 vars in 1365 bytes} [Thu May 19 12:24:57 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 12:25:03.140525 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5cec5b-b5ea-4bd4-b3eb-f1b6f1fdd9d1 None None] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:25:03.152426 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5cec5b-b5ea-4bd4-b3eb-f1b6f1fdd9d1 None None] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:25:03.156696 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0754dafb-4b08-438e-bfa7-7e00919cf1ee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a00b3fa7497456895e5e0bf105c4fc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd70d25bf69314820b8aaecb65576978a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:25:03.157355 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0754dafb-4b08-438e-bfa7-7e00919cf1ee service neutron] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:25:03.157640 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0754dafb-4b08-438e-bfa7-7e00919cf1ee service neutron] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:25:03.157916 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0754dafb-4b08-438e-bfa7-7e00919cf1ee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:25:03.165386 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-0754dafb-4b08-438e-bfa7-7e00919cf1ee service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:25:03.233486 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0754dafb-4b08-438e-bfa7-7e00919cf1ee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:25:03.235082 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0754dafb-4b08-438e-bfa7-7e00919cf1ee service neutron] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:25:03.247325 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1144/2309] 10.222.0.23 () {60 vars in 1365 bytes} [Thu May 19 12:25:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:25:07.048328 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=6 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:25:08.183055 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0766685b-54f9-4ad5-86f2-4229974fa9c4 service neutron] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:25:08.184821 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-0766685b-54f9-4ad5-86f2-4229974fa9c4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:25:08.203382 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0766685b-54f9-4ad5-86f2-4229974fa9c4 service neutron] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:25:08.210251 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3a5ec8-72fc-4a75-a501-b4f19b4415c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:25:08.210251 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3a5ec8-72fc-4a75-a501-b4f19b4415c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:25:08.210251 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3a5ec8-72fc-4a75-a501-b4f19b4415c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:25:08.210251 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3a5ec8-72fc-4a75-a501-b4f19b4415c5 admin admin] PATH_INFO: `/v3/users/8e445569dc5641ed962b0ac977850a6c` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:25:08.211018 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3a5ec8-72fc-4a75-a501-b4f19b4415c5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8e445569dc5641ed962b0ac977850a6c)` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:25:08.212759 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3a5ec8-72fc-4a75-a501-b4f19b4415c5 admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:25:08.485505 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-3d3a5ec8-72fc-4a75-a501-b4f19b4415c5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8e445569dc5641ed962b0ac977850a6c'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:25:08.498086 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-3d3a5ec8-72fc-4a75-a501-b4f19b4415c5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8e445569dc5641ed962b0ac977850a6c'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:25:08.498770 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-3d3a5ec8-72fc-4a75-a501-b4f19b4415c5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8e445569dc5641ed962b0ac977850a6c'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:25:08.590168 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-3d3a5ec8-72fc-4a75-a501-b4f19b4415c5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8e445569dc5641ed962b0ac977850a6c'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:25:08.598974 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-3d3a5ec8-72fc-4a75-a501-b4f19b4415c5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8e445569dc5641ed962b0ac977850a6c'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:25:08.605371 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1166/2310] 10.222.0.23 () {58 vars in 1223 bytes} [Thu May 19 12:25:08 2022] DELETE /identity/v3/users/8e445569dc5641ed962b0ac977850a6c => generated 0 bytes in 423 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:25:08.940441 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0754dafb-4b08-438e-bfa7-7e00919cf1ee service neutron] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:25:08.951467 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0754dafb-4b08-438e-bfa7-7e00919cf1ee service neutron] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:25:09.020418 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47565fd9-1fa4-4f88-94d0-fd2b356dc778 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:25:09.021502 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47565fd9-1fa4-4f88-94d0-fd2b356dc778 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:25:09.021782 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47565fd9-1fa4-4f88-94d0-fd2b356dc778 admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:25:09.022056 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47565fd9-1fa4-4f88-94d0-fd2b356dc778 admin admin] PATH_INFO: `/v3/projects/b3d651a605fd4dee95797a7ec3896792` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:25:09.023270 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47565fd9-1fa4-4f88-94d0-fd2b356dc778 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b3d651a605fd4dee95797a7ec3896792)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:25:09.025241 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47565fd9-1fa4-4f88-94d0-fd2b356dc778 admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:25:09.262635 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-47565fd9-1fa4-4f88-94d0-fd2b356dc778 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b3d651a605fd4dee95797a7ec3896792', 'request_id': 'req-47565fd9-1fa4-4f88-94d0-fd2b356dc778'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:25:09.269751 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-47565fd9-1fa4-4f88-94d0-fd2b356dc778 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b3d651a605fd4dee95797a7ec3896792', 'request_id': 'req-47565fd9-1fa4-4f88-94d0-fd2b356dc778'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:25:09.376052 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-47565fd9-1fa4-4f88-94d0-fd2b356dc778 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b3d651a605fd4dee95797a7ec3896792', 'request_id': 'req-47565fd9-1fa4-4f88-94d0-fd2b356dc778'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:25:09.387565 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-47565fd9-1fa4-4f88-94d0-fd2b356dc778 admin admin] The token cache is being invalidate because project b3d651a605fd4dee95797a7ec3896792 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80539) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 12:25:09.387899 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-47565fd9-1fa4-4f88-94d0-fd2b356dc778 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:25:09.388544 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-47565fd9-1fa4-4f88-94d0-fd2b356dc778 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:25:09.390086 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1145/2311] 10.222.0.23 () {58 vars in 1232 bytes} [Thu May 19 12:25:08 2022] DELETE /identity/v3/projects/b3d651a605fd4dee95797a7ec3896792 => generated 0 bytes in 450 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:25:09.395298 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7368ba8b-840d-49fa-a49a-b5a346bacd1c None None] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:25:09.395881 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7368ba8b-840d-49fa-a49a-b5a346bacd1c None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:25:09.395981 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7368ba8b-840d-49fa-a49a-b5a346bacd1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:25:09.414804 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.auth.core [None req-7368ba8b-840d-49fa-a49a-b5a346bacd1c None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:25:09.416044 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-7368ba8b-840d-49fa-a49a-b5a346bacd1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:25:09.476449 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1167/2312] 10.222.0.23 () {58 vars in 957 bytes} [Thu May 19 12:25:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:25:09.484449 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a114475-5ee2-4bfd-8ad0-7628e84b7810 None None] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:25:09.484740 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a114475-5ee2-4bfd-8ad0-7628e84b7810 None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:25:09.485017 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a114475-5ee2-4bfd-8ad0-7628e84b7810 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:25:09.504989 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.auth.core [None req-9a114475-5ee2-4bfd-8ad0-7628e84b7810 None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:25:09.506212 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-9a114475-5ee2-4bfd-8ad0-7628e84b7810 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:25:09.514207 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1146/2313] 10.222.0.23 () {58 vars in 957 bytes} [Thu May 19 12:25:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:25:09.518563 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7368ba8b-840d-49fa-a49a-b5a346bacd1c None None] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:25:09.528895 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7368ba8b-840d-49fa-a49a-b5a346bacd1c None None] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:25:09.533725 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0714b3ea-e964-4124-8a48-86db560d1bac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:25:09.534588 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0714b3ea-e964-4124-8a48-86db560d1bac admin admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:25:09.535082 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0714b3ea-e964-4124-8a48-86db560d1bac admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:25:09.535510 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0714b3ea-e964-4124-8a48-86db560d1bac admin admin] PATH_INFO: `/v3/domains` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:25:09.536289 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0714b3ea-e964-4124-8a48-86db560d1bac admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:25:09.536595 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0714b3ea-e964-4124-8a48-86db560d1bac admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:25:09.538270 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0714b3ea-e964-4124-8a48-86db560d1bac admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:25:09.552112 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1168/2314] 10.222.0.23 () {58 vars in 1152 bytes} [Thu May 19 12:25:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:25:09.556914 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a114475-5ee2-4bfd-8ad0-7628e84b7810 None None] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:25:09.568753 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a114475-5ee2-4bfd-8ad0-7628e84b7810 None None] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:25:09.573466 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1e7e07-29e1-472b-b593-221790c30122 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:25:09.574180 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1e7e07-29e1-472b-b593-221790c30122 admin admin] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:25:09.574461 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1e7e07-29e1-472b-b593-221790c30122 admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:25:09.574734 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1e7e07-29e1-472b-b593-221790c30122 admin admin] PATH_INFO: `/v3/projects` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:25:09.575436 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1e7e07-29e1-472b-b593-221790c30122 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:25:09.577387 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1e7e07-29e1-472b-b593-221790c30122 admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:25:09.586257 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:25:09.694680 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1147/2315] 10.222.0.23 () {60 vars in 1152 bytes} [Thu May 19 12:25:09 2022] POST /identity/v3/projects => generated 383 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 12:25:09.698956 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0714b3ea-e964-4124-8a48-86db560d1bac admin admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:25:09.709052 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0714b3ea-e964-4124-8a48-86db560d1bac admin admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:25:09.766046 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aedf80c-1af4-4663-8985-6afc0d728990 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:25:09.766629 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aedf80c-1af4-4663-8985-6afc0d728990 admin admin] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:25:09.766914 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aedf80c-1af4-4663-8985-6afc0d728990 admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:25:09.767186 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aedf80c-1af4-4663-8985-6afc0d728990 admin admin] PATH_INFO: `/v3/users` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:25:09.767903 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aedf80c-1af4-4663-8985-6afc0d728990 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:25:09.769879 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aedf80c-1af4-4663-8985-6afc0d728990 admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:25:09.776474 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:25:09.957085 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1169/2316] 10.222.0.23 () {60 vars in 1143 bytes} [Thu May 19 12:25:09 2022] POST /identity/v3/users => generated 292 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 12:25:09.960966 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1e7e07-29e1-472b-b593-221790c30122 admin admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:25:09.971244 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1e7e07-29e1-472b-b593-221790c30122 admin admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:25:09.976574 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2840af8b-095b-40e3-96cc-b5c17040cb31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:25:09.977233 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2840af8b-095b-40e3-96cc-b5c17040cb31 admin admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:25:09.977520 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2840af8b-095b-40e3-96cc-b5c17040cb31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:25:09.977800 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2840af8b-095b-40e3-96cc-b5c17040cb31 admin admin] PATH_INFO: `/v3/roles` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:25:09.978534 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2840af8b-095b-40e3-96cc-b5c17040cb31 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:25:09.980179 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2840af8b-095b-40e3-96cc-b5c17040cb31 admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:25:09.990662 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1148/2317] 10.222.0.23 () {58 vars in 1121 bytes} [Thu May 19 12:25:09 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 12:25:09.994802 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aedf80c-1af4-4663-8985-6afc0d728990 admin admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:25:10.005632 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aedf80c-1af4-4663-8985-6afc0d728990 admin admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:25:10.010408 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e436560-14ff-4339-a882-7de23072975b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:25:10.011027 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e436560-14ff-4339-a882-7de23072975b admin admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:25:10.032121 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e436560-14ff-4339-a882-7de23072975b admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:25:10.032121 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e436560-14ff-4339-a882-7de23072975b admin admin] PATH_INFO: `/v3/roles` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:25:10.032121 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e436560-14ff-4339-a882-7de23072975b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:25:10.032121 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e436560-14ff-4339-a882-7de23072975b admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:25:10.032121 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1170/2318] 10.222.0.23 () {58 vars in 1121 bytes} [Thu May 19 12:25:09 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 12:25:10.035498 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2840af8b-095b-40e3-96cc-b5c17040cb31 admin admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:25:10.045427 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2840af8b-095b-40e3-96cc-b5c17040cb31 admin admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:25:10.049965 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d442d289-c958-4b52-993b-72251c9098f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:25:10.051229 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d442d289-c958-4b52-993b-72251c9098f6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:25:10.051366 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d442d289-c958-4b52-993b-72251c9098f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:25:10.051566 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d442d289-c958-4b52-993b-72251c9098f6 admin admin] PATH_INFO: `/v3/projects/54a9930dd8cf4efdbd257f9df7aecbce/users/d8e636431a2e46c7aed3776e50b04523/roles/259eb400cdb4439db72d14456ae9cd08` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:25:10.073791 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d442d289-c958-4b52-993b-72251c9098f6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=54a9930dd8cf4efdbd257f9df7aecbce, user_id=d8e636431a2e46c7aed3776e50b04523, role_id=259eb400cdb4439db72d14456ae9cd08)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:25:10.083574 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d442d289-c958-4b52-993b-72251c9098f6 admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:25:10.217830 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1149/2319] 10.222.0.23 () {60 vars in 1482 bytes} [Thu May 19 12:25:10 2022] PUT /identity/v3/projects/54a9930dd8cf4efdbd257f9df7aecbce/users/d8e636431a2e46c7aed3776e50b04523/roles/259eb400cdb4439db72d14456ae9cd08 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:25:10.229579 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e600e673-d290-4a84-b28b-ce45e818932e None None] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:25:10.229866 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e600e673-d290-4a84-b28b-ce45e818932e None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:25:10.230146 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e600e673-d290-4a84-b28b-ce45e818932e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:25:10.260082 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.auth.core [None req-e600e673-d290-4a84-b28b-ce45e818932e None None] MFA Rules not processed for user `d8e636431a2e46c7aed3776e50b04523`. Rule list: `[]` (Enabled: `True`). {{(pid=80540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:25:10.261519 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-e600e673-d290-4a84-b28b-ce45e818932e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:25:10.347655 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1171/2320] 10.222.0.23 () {58 vars in 957 bytes} [Thu May 19 12:25:10 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:25:10.418717 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d442d289-c958-4b52-993b-72251c9098f6 admin admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:25:10.421338 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-d442d289-c958-4b52-993b-72251c9098f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:25:10.501654 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d442d289-c958-4b52-993b-72251c9098f6 admin admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:25:10.508028 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d73ff9-b5d6-438b-a9ed-ced4230d80d5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34a98d75d84b4c67981377e0f26eca62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd70d25bf69314820b8aaecb65576978a', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:25:10.508779 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d73ff9-b5d6-438b-a9ed-ced4230d80d5 service nova] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:25:10.509135 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d73ff9-b5d6-438b-a9ed-ced4230d80d5 service nova] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:25:10.509486 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d73ff9-b5d6-438b-a9ed-ced4230d80d5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:25:10.518484 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d73ff9-b5d6-438b-a9ed-ced4230d80d5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:25:10.521068 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d73ff9-b5d6-438b-a9ed-ced4230d80d5 service nova] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:25:10.532389 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1150/2321] 10.222.0.23 () {60 vars in 1365 bytes} [Thu May 19 12:25:10 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:25:18.878837 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e600e673-d290-4a84-b28b-ce45e818932e None None] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:25:18.892255 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e600e673-d290-4a84-b28b-ce45e818932e None None] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:25:18.898319 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1f7c30-4151-4ee9-90af-ca94f8cea6a6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34a98d75d84b4c67981377e0f26eca62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd70d25bf69314820b8aaecb65576978a', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:25:18.898985 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1f7c30-4151-4ee9-90af-ca94f8cea6a6 service nova] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:25:18.899272 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1f7c30-4151-4ee9-90af-ca94f8cea6a6 service nova] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:25:18.899556 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1f7c30-4151-4ee9-90af-ca94f8cea6a6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:25:18.902226 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-1f1f7c30-4151-4ee9-90af-ca94f8cea6a6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:25:18.995969 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1f7c30-4151-4ee9-90af-ca94f8cea6a6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:25:18.997550 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1f7c30-4151-4ee9-90af-ca94f8cea6a6 service nova] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:25:19.009580 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1172/2322] 10.222.0.23 () {60 vars in 1365 bytes} [Thu May 19 12:25:18 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:25:20.527149 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=57,keystone:INSERT=2 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:25:22.312766 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d73ff9-b5d6-438b-a9ed-ced4230d80d5 service nova] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:25:22.375138 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d73ff9-b5d6-438b-a9ed-ced4230d80d5 service nova] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:25:22.380206 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1162b6c-0b5b-4d27-bae8-141126865f90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:25:22.381067 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1162b6c-0b5b-4d27-bae8-141126865f90 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:25:22.381351 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1162b6c-0b5b-4d27-bae8-141126865f90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:25:22.381622 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1162b6c-0b5b-4d27-bae8-141126865f90 admin admin] PATH_INFO: `/v3/users/d8e636431a2e46c7aed3776e50b04523` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:25:22.382902 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1162b6c-0b5b-4d27-bae8-141126865f90 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d8e636431a2e46c7aed3776e50b04523)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:25:22.384860 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1162b6c-0b5b-4d27-bae8-141126865f90 admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:25:22.649231 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-e1162b6c-0b5b-4d27-bae8-141126865f90 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd8e636431a2e46c7aed3776e50b04523'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:25:22.661803 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-e1162b6c-0b5b-4d27-bae8-141126865f90 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd8e636431a2e46c7aed3776e50b04523'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:25:22.662478 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-e1162b6c-0b5b-4d27-bae8-141126865f90 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd8e636431a2e46c7aed3776e50b04523'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:25:22.747009 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-e1162b6c-0b5b-4d27-bae8-141126865f90 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd8e636431a2e46c7aed3776e50b04523'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:25:22.755648 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-e1162b6c-0b5b-4d27-bae8-141126865f90 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd8e636431a2e46c7aed3776e50b04523'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:25:22.761932 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1151/2323] 10.222.0.23 () {58 vars in 1223 bytes} [Thu May 19 12:25:22 2022] DELETE /identity/v3/users/d8e636431a2e46c7aed3776e50b04523 => generated 0 bytes in 450 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:25:22.766552 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05858bcf-156d-4992-9a9b-9e40b35ba512 None None] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:25:22.767046 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05858bcf-156d-4992-9a9b-9e40b35ba512 None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:25:22.767240 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05858bcf-156d-4992-9a9b-9e40b35ba512 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:25:22.788963 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.auth.core [None req-05858bcf-156d-4992-9a9b-9e40b35ba512 None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:25:22.791159 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-05858bcf-156d-4992-9a9b-9e40b35ba512 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:25:22.851918 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1173/2324] 10.222.0.23 () {58 vars in 957 bytes} [Thu May 19 12:25:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:25:22.861673 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1162b6c-0b5b-4d27-bae8-141126865f90 admin admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:25:22.863377 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-e1162b6c-0b5b-4d27-bae8-141126865f90 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:25:22.928581 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1162b6c-0b5b-4d27-bae8-141126865f90 admin admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:25:22.928581 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499a2d3e-5c38-4b31-a441-5c9cfb5d4856 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a00b3fa7497456895e5e0bf105c4fc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd70d25bf69314820b8aaecb65576978a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:25:22.928942 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499a2d3e-5c38-4b31-a441-5c9cfb5d4856 service neutron] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:25:22.929197 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499a2d3e-5c38-4b31-a441-5c9cfb5d4856 service neutron] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:25:22.929473 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499a2d3e-5c38-4b31-a441-5c9cfb5d4856 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:25:22.938902 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499a2d3e-5c38-4b31-a441-5c9cfb5d4856 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:25:22.941027 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499a2d3e-5c38-4b31-a441-5c9cfb5d4856 service neutron] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:25:22.951169 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1152/2325] 10.222.0.23 () {60 vars in 1365 bytes} [Thu May 19 12:25:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:25:23.472862 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05858bcf-156d-4992-9a9b-9e40b35ba512 None None] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:25:23.479592 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-05858bcf-156d-4992-9a9b-9e40b35ba512 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:25:23.486721 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05858bcf-156d-4992-9a9b-9e40b35ba512 None None] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:25:23.491119 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164bd03b-717e-404c-8e77-c6dfdeeae81d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a00b3fa7497456895e5e0bf105c4fc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd70d25bf69314820b8aaecb65576978a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:25:23.491670 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164bd03b-717e-404c-8e77-c6dfdeeae81d service neutron] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:25:23.491991 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164bd03b-717e-404c-8e77-c6dfdeeae81d service neutron] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:25:23.492278 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164bd03b-717e-404c-8e77-c6dfdeeae81d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:25:23.494205 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-164bd03b-717e-404c-8e77-c6dfdeeae81d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:25:23.501293 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:25:23.505649 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164bd03b-717e-404c-8e77-c6dfdeeae81d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:25:23.507187 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164bd03b-717e-404c-8e77-c6dfdeeae81d service neutron] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:25:23.516461 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1174/2326] 10.222.0.23 () {60 vars in 1365 bytes} [Thu May 19 12:25:23 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:25:23.717345 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499a2d3e-5c38-4b31-a441-5c9cfb5d4856 service neutron] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:25:23.718765 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-499a2d3e-5c38-4b31-a441-5c9cfb5d4856 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:25:23.734364 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499a2d3e-5c38-4b31-a441-5c9cfb5d4856 service neutron] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:25:23.739362 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5e29d0-807f-4264-b6f8-8fd59ca4ee23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:25:23.740360 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5e29d0-807f-4264-b6f8-8fd59ca4ee23 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:25:23.745694 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5e29d0-807f-4264-b6f8-8fd59ca4ee23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:25:23.745694 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5e29d0-807f-4264-b6f8-8fd59ca4ee23 admin admin] PATH_INFO: `/v3/projects/54a9930dd8cf4efdbd257f9df7aecbce` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:25:23.745694 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5e29d0-807f-4264-b6f8-8fd59ca4ee23 admin admin] RBAC: Authorizing `identity:delete_project(project_id=54a9930dd8cf4efdbd257f9df7aecbce)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:25:23.745694 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5e29d0-807f-4264-b6f8-8fd59ca4ee23 admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:25:23.767084 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:25:23.936287 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-6e5e29d0-807f-4264-b6f8-8fd59ca4ee23 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '54a9930dd8cf4efdbd257f9df7aecbce', 'request_id': 'req-6e5e29d0-807f-4264-b6f8-8fd59ca4ee23'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:25:23.944024 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-6e5e29d0-807f-4264-b6f8-8fd59ca4ee23 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '54a9930dd8cf4efdbd257f9df7aecbce', 'request_id': 'req-6e5e29d0-807f-4264-b6f8-8fd59ca4ee23'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:25:24.035922 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-6e5e29d0-807f-4264-b6f8-8fd59ca4ee23 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '54a9930dd8cf4efdbd257f9df7aecbce', 'request_id': 'req-6e5e29d0-807f-4264-b6f8-8fd59ca4ee23'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:25:24.050368 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-6e5e29d0-807f-4264-b6f8-8fd59ca4ee23 admin admin] The token cache is being invalidate because project 54a9930dd8cf4efdbd257f9df7aecbce was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80539) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 12:25:24.050692 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-6e5e29d0-807f-4264-b6f8-8fd59ca4ee23 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:25:24.051335 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-6e5e29d0-807f-4264-b6f8-8fd59ca4ee23 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:25:24.053244 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1153/2327] 10.222.0.23 () {58 vars in 1232 bytes} [Thu May 19 12:25:23 2022] DELETE /identity/v3/projects/54a9930dd8cf4efdbd257f9df7aecbce => generated 0 bytes in 336 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:25:24.058243 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a837ee-8031-4760-b2a7-814afa33483c None None] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:25:24.058738 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a837ee-8031-4760-b2a7-814afa33483c None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:25:24.058844 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a837ee-8031-4760-b2a7-814afa33483c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:25:24.081809 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.auth.core [None req-63a837ee-8031-4760-b2a7-814afa33483c None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:25:24.083093 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-63a837ee-8031-4760-b2a7-814afa33483c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:25:24.173478 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1175/2328] 10.222.0.23 () {58 vars in 957 bytes} [Thu May 19 12:25:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:25:24.176960 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904cb595-5978-4347-aec4-26d731866f66 None None] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:25:24.177471 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904cb595-5978-4347-aec4-26d731866f66 None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:25:24.177840 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904cb595-5978-4347-aec4-26d731866f66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:25:24.211337 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.auth.core [None req-904cb595-5978-4347-aec4-26d731866f66 None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:25:24.212670 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-904cb595-5978-4347-aec4-26d731866f66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:25:24.232088 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1154/2329] 10.222.0.23 () {58 vars in 957 bytes} [Thu May 19 12:25:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:25:24.232177 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a837ee-8031-4760-b2a7-814afa33483c None None] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:25:24.255733 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a837ee-8031-4760-b2a7-814afa33483c None None] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:25:24.261260 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20cb3673-cd44-46c6-8028-3274a7c52187 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:25:24.262145 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cb3673-cd44-46c6-8028-3274a7c52187 admin admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:25:24.262622 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cb3673-cd44-46c6-8028-3274a7c52187 admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:25:24.262780 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cb3673-cd44-46c6-8028-3274a7c52187 admin admin] PATH_INFO: `/v3/domains` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:25:24.264399 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20cb3673-cd44-46c6-8028-3274a7c52187 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:25:24.264399 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20cb3673-cd44-46c6-8028-3274a7c52187 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:25:24.265616 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20cb3673-cd44-46c6-8028-3274a7c52187 admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:25:24.279364 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1176/2330] 10.222.0.23 () {58 vars in 1152 bytes} [Thu May 19 12:25:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:25:24.283663 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904cb595-5978-4347-aec4-26d731866f66 None None] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:25:24.293548 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904cb595-5978-4347-aec4-26d731866f66 None None] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:25:24.298589 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef456152-db16-4e64-8126-e5f4383e640d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:25:24.299255 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef456152-db16-4e64-8126-e5f4383e640d admin admin] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:25:24.300232 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef456152-db16-4e64-8126-e5f4383e640d admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:25:24.301118 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef456152-db16-4e64-8126-e5f4383e640d admin admin] PATH_INFO: `/v3/projects` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:25:24.301822 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef456152-db16-4e64-8126-e5f4383e640d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:25:24.303984 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef456152-db16-4e64-8126-e5f4383e640d admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:25:24.460981 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1155/2331] 10.222.0.23 () {60 vars in 1152 bytes} [Thu May 19 12:25:24 2022] POST /identity/v3/projects => generated 373 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 12:25:24.480568 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20cb3673-cd44-46c6-8028-3274a7c52187 admin admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:25:24.480568 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20cb3673-cd44-46c6-8028-3274a7c52187 admin admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:25:24.547376 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d5e67d-2e72-4bc6-afd4-da644e68b2d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:25:24.548054 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d5e67d-2e72-4bc6-afd4-da644e68b2d8 admin admin] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:25:24.548343 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d5e67d-2e72-4bc6-afd4-da644e68b2d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:25:24.548622 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d5e67d-2e72-4bc6-afd4-da644e68b2d8 admin admin] PATH_INFO: `/v3/users` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:25:24.549324 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d5e67d-2e72-4bc6-afd4-da644e68b2d8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:25:24.552463 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d5e67d-2e72-4bc6-afd4-da644e68b2d8 admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:25:24.679594 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1177/2332] 10.222.0.23 () {60 vars in 1143 bytes} [Thu May 19 12:25:24 2022] POST /identity/v3/users => generated 287 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 12:25:24.683653 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef456152-db16-4e64-8126-e5f4383e640d admin admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:25:24.695705 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef456152-db16-4e64-8126-e5f4383e640d admin admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:25:24.700482 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ebd6c4-e937-41a6-8b7b-d83c82de601a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:25:24.701149 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ebd6c4-e937-41a6-8b7b-d83c82de601a admin admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:25:24.701431 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ebd6c4-e937-41a6-8b7b-d83c82de601a admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:25:24.701704 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ebd6c4-e937-41a6-8b7b-d83c82de601a admin admin] PATH_INFO: `/v3/roles` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:25:24.702627 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ebd6c4-e937-41a6-8b7b-d83c82de601a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:25:24.704410 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ebd6c4-e937-41a6-8b7b-d83c82de601a admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:25:24.716463 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1156/2333] 10.222.0.23 () {58 vars in 1121 bytes} [Thu May 19 12:25:24 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 12:25:24.720560 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d5e67d-2e72-4bc6-afd4-da644e68b2d8 admin admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:25:24.732760 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d5e67d-2e72-4bc6-afd4-da644e68b2d8 admin admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:25:24.738648 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7a9b9a-dd8a-41bd-9d9b-fdcac64db2a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:25:24.739473 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7a9b9a-dd8a-41bd-9d9b-fdcac64db2a0 admin admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:25:24.739894 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7a9b9a-dd8a-41bd-9d9b-fdcac64db2a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:25:24.740250 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7a9b9a-dd8a-41bd-9d9b-fdcac64db2a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:25:24.741094 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a7a9b9a-dd8a-41bd-9d9b-fdcac64db2a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:25:24.742935 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a7a9b9a-dd8a-41bd-9d9b-fdcac64db2a0 admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:25:24.754403 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1178/2334] 10.222.0.23 () {58 vars in 1121 bytes} [Thu May 19 12:25:24 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 19 12:25:24.758662 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ebd6c4-e937-41a6-8b7b-d83c82de601a admin admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:25:24.770521 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ebd6c4-e937-41a6-8b7b-d83c82de601a admin admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:25:24.774535 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38095a4-6481-4db2-a6ca-df9449ef9b1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:25:24.775418 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38095a4-6481-4db2-a6ca-df9449ef9b1a admin admin] REQUEST_METHOD: `PUT` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:25:24.775693 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38095a4-6481-4db2-a6ca-df9449ef9b1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:25:24.776052 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38095a4-6481-4db2-a6ca-df9449ef9b1a admin admin] PATH_INFO: `/v3/projects/246503f8f9214c469e1fe0e4ad5957bf/users/880a01daf5294d259744e7b330e142b6/roles/259eb400cdb4439db72d14456ae9cd08` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:25:24.791586 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38095a4-6481-4db2-a6ca-df9449ef9b1a admin admin] RBAC: Authorizing `identity:create_grant(project_id=246503f8f9214c469e1fe0e4ad5957bf, user_id=880a01daf5294d259744e7b330e142b6, role_id=259eb400cdb4439db72d14456ae9cd08)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:25:24.794366 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38095a4-6481-4db2-a6ca-df9449ef9b1a admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:25:24.903933 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1157/2335] 10.222.0.23 () {60 vars in 1482 bytes} [Thu May 19 12:25:24 2022] PUT /identity/v3/projects/246503f8f9214c469e1fe0e4ad5957bf/users/880a01daf5294d259744e7b330e142b6/roles/259eb400cdb4439db72d14456ae9cd08 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:25:24.915132 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6880a330-0fa7-4ef2-ac1a-9b4044de0919 None None] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:25:24.915442 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6880a330-0fa7-4ef2-ac1a-9b4044de0919 None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:25:24.915709 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6880a330-0fa7-4ef2-ac1a-9b4044de0919 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:25:24.944093 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.auth.core [None req-6880a330-0fa7-4ef2-ac1a-9b4044de0919 None None] MFA Rules not processed for user `880a01daf5294d259744e7b330e142b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:25:24.945319 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-6880a330-0fa7-4ef2-ac1a-9b4044de0919 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:25:24.996269 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:25:25.014450 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1179/2336] 10.222.0.23 () {58 vars in 957 bytes} [Thu May 19 12:25:24 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:25:25.025288 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38095a4-6481-4db2-a6ca-df9449ef9b1a admin admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:25:25.027132 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-a38095a4-6481-4db2-a6ca-df9449ef9b1a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:25:25.105190 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38095a4-6481-4db2-a6ca-df9449ef9b1a admin admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:25:25.111227 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d0a14e-8b85-4e3b-92ea-f556d159ffe8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34a98d75d84b4c67981377e0f26eca62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd70d25bf69314820b8aaecb65576978a', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:25:25.112428 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d0a14e-8b85-4e3b-92ea-f556d159ffe8 service nova] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:25:25.113036 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d0a14e-8b85-4e3b-92ea-f556d159ffe8 service nova] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:25:25.113036 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d0a14e-8b85-4e3b-92ea-f556d159ffe8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:25:25.121279 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d0a14e-8b85-4e3b-92ea-f556d159ffe8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:25:25.123035 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d0a14e-8b85-4e3b-92ea-f556d159ffe8 service nova] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:25:25.133501 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1158/2337] 10.222.0.23 () {60 vars in 1365 bytes} [Thu May 19 12:25:25 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:25:29.633290 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6880a330-0fa7-4ef2-ac1a-9b4044de0919 None None] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:25:29.694763 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6880a330-0fa7-4ef2-ac1a-9b4044de0919 None None] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:25:29.699534 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363c4455-cd33-4f86-89b9-61b8097963cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:25:29.700519 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363c4455-cd33-4f86-89b9-61b8097963cd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:25:29.700834 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363c4455-cd33-4f86-89b9-61b8097963cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:25:29.701109 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363c4455-cd33-4f86-89b9-61b8097963cd admin admin] PATH_INFO: `/v3/users/880a01daf5294d259744e7b330e142b6` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:25:29.702306 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363c4455-cd33-4f86-89b9-61b8097963cd admin admin] RBAC: Authorizing `identity:delete_user(user_id=880a01daf5294d259744e7b330e142b6)` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:25:29.704913 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363c4455-cd33-4f86-89b9-61b8097963cd admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:25:29.966957 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-363c4455-cd33-4f86-89b9-61b8097963cd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '880a01daf5294d259744e7b330e142b6'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:25:29.979141 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-363c4455-cd33-4f86-89b9-61b8097963cd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '880a01daf5294d259744e7b330e142b6'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:25:29.979841 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-363c4455-cd33-4f86-89b9-61b8097963cd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '880a01daf5294d259744e7b330e142b6'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:25:30.047392 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-363c4455-cd33-4f86-89b9-61b8097963cd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '880a01daf5294d259744e7b330e142b6'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:25:30.055562 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-363c4455-cd33-4f86-89b9-61b8097963cd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '880a01daf5294d259744e7b330e142b6'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:25:30.062050 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1180/2338] 10.222.0.23 () {58 vars in 1223 bytes} [Thu May 19 12:25:29 2022] DELETE /identity/v3/users/880a01daf5294d259744e7b330e142b6 => generated 0 bytes in 431 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:25:30.067510 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e124a45-d3fa-420f-9639-b398d9366bb9 None None] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:25:30.068043 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e124a45-d3fa-420f-9639-b398d9366bb9 None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:25:30.068166 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e124a45-d3fa-420f-9639-b398d9366bb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:25:30.087753 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.auth.core [None req-4e124a45-d3fa-420f-9639-b398d9366bb9 None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:25:30.089045 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-4e124a45-d3fa-420f-9639-b398d9366bb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:25:30.111698 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:25:30.153473 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1159/2339] 10.222.0.23 () {58 vars in 957 bytes} [Thu May 19 12:25:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:25:30.163690 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363c4455-cd33-4f86-89b9-61b8097963cd admin admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:25:30.165253 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-363c4455-cd33-4f86-89b9-61b8097963cd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:25:30.232755 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363c4455-cd33-4f86-89b9-61b8097963cd admin admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:25:30.236630 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370950dc-01c8-4501-b7d5-2bbd0a219473 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a00b3fa7497456895e5e0bf105c4fc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd70d25bf69314820b8aaecb65576978a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:25:30.238007 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370950dc-01c8-4501-b7d5-2bbd0a219473 service neutron] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:25:30.238373 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370950dc-01c8-4501-b7d5-2bbd0a219473 service neutron] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:25:30.238574 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370950dc-01c8-4501-b7d5-2bbd0a219473 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:25:30.248514 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370950dc-01c8-4501-b7d5-2bbd0a219473 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:25:30.249976 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370950dc-01c8-4501-b7d5-2bbd0a219473 service neutron] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:25:30.261856 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1181/2340] 10.222.0.23 () {60 vars in 1365 bytes} [Thu May 19 12:25:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:25:30.753391 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e124a45-d3fa-420f-9639-b398d9366bb9 None None] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:25:30.755238 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-4e124a45-d3fa-420f-9639-b398d9366bb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:25:30.769891 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e124a45-d3fa-420f-9639-b398d9366bb9 None None] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:25:30.775278 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d70d121-7c21-4069-bd5d-40487a28a4e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:25:30.776215 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d70d121-7c21-4069-bd5d-40487a28a4e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:25:30.776499 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d70d121-7c21-4069-bd5d-40487a28a4e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:25:30.776913 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d70d121-7c21-4069-bd5d-40487a28a4e9 admin admin] PATH_INFO: `/v3/projects/246503f8f9214c469e1fe0e4ad5957bf` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:25:30.778238 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d70d121-7c21-4069-bd5d-40487a28a4e9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=246503f8f9214c469e1fe0e4ad5957bf)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:25:30.780238 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d70d121-7c21-4069-bd5d-40487a28a4e9 admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:25:30.896186 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-5d70d121-7c21-4069-bd5d-40487a28a4e9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '246503f8f9214c469e1fe0e4ad5957bf', 'request_id': 'req-5d70d121-7c21-4069-bd5d-40487a28a4e9'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:25:30.902328 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-5d70d121-7c21-4069-bd5d-40487a28a4e9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '246503f8f9214c469e1fe0e4ad5957bf', 'request_id': 'req-5d70d121-7c21-4069-bd5d-40487a28a4e9'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:25:31.015890 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-5d70d121-7c21-4069-bd5d-40487a28a4e9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '246503f8f9214c469e1fe0e4ad5957bf', 'request_id': 'req-5d70d121-7c21-4069-bd5d-40487a28a4e9'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:25:31.028991 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-5d70d121-7c21-4069-bd5d-40487a28a4e9 admin admin] The token cache is being invalidate because project 246503f8f9214c469e1fe0e4ad5957bf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80539) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 12:25:31.029281 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-5d70d121-7c21-4069-bd5d-40487a28a4e9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:25:31.030020 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-5d70d121-7c21-4069-bd5d-40487a28a4e9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:25:31.031670 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1160/2341] 10.222.0.23 () {58 vars in 1232 bytes} [Thu May 19 12:25:30 2022] DELETE /identity/v3/projects/246503f8f9214c469e1fe0e4ad5957bf => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:25:31.037642 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce7a358-3e29-4bbb-9c73-6dfc282659c6 None None] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:25:31.038177 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce7a358-3e29-4bbb-9c73-6dfc282659c6 None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:25:31.038177 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce7a358-3e29-4bbb-9c73-6dfc282659c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:25:31.057184 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.auth.core [None req-dce7a358-3e29-4bbb-9c73-6dfc282659c6 None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:25:31.058555 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-dce7a358-3e29-4bbb-9c73-6dfc282659c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:25:31.064297 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:25:31.120352 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1182/2342] 10.222.0.23 () {58 vars in 957 bytes} [Thu May 19 12:25:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:25:31.128376 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de267c2d-ac7f-4853-b589-0e5b657d3984 None None] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:25:31.128882 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de267c2d-ac7f-4853-b589-0e5b657d3984 None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:25:31.129001 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de267c2d-ac7f-4853-b589-0e5b657d3984 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:25:31.150271 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.auth.core [None req-de267c2d-ac7f-4853-b589-0e5b657d3984 None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:25:31.151495 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-de267c2d-ac7f-4853-b589-0e5b657d3984 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:25:31.160306 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1161/2343] 10.222.0.23 () {58 vars in 957 bytes} [Thu May 19 12:25:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:25:31.164666 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce7a358-3e29-4bbb-9c73-6dfc282659c6 None None] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:25:31.177873 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce7a358-3e29-4bbb-9c73-6dfc282659c6 None None] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:25:31.182761 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a045bea-bce3-453d-a017-60be3c8b2da9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:25:31.183595 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a045bea-bce3-453d-a017-60be3c8b2da9 admin admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:25:31.183899 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a045bea-bce3-453d-a017-60be3c8b2da9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:25:31.184203 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a045bea-bce3-453d-a017-60be3c8b2da9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:25:31.184975 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a045bea-bce3-453d-a017-60be3c8b2da9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:25:31.185429 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a045bea-bce3-453d-a017-60be3c8b2da9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:25:31.187098 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a045bea-bce3-453d-a017-60be3c8b2da9 admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:25:31.201043 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1183/2344] 10.222.0.23 () {58 vars in 1152 bytes} [Thu May 19 12:25:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:25:31.207017 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de267c2d-ac7f-4853-b589-0e5b657d3984 None None] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:25:31.216875 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de267c2d-ac7f-4853-b589-0e5b657d3984 None None] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:25:31.222359 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42592a3f-2530-46a0-aad4-050563eb39a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:25:31.223026 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42592a3f-2530-46a0-aad4-050563eb39a6 admin admin] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:25:31.223315 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42592a3f-2530-46a0-aad4-050563eb39a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:25:31.223596 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42592a3f-2530-46a0-aad4-050563eb39a6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:25:31.224342 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42592a3f-2530-46a0-aad4-050563eb39a6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:25:31.226459 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42592a3f-2530-46a0-aad4-050563eb39a6 admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:25:31.359247 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1162/2345] 10.222.0.23 () {60 vars in 1152 bytes} [Thu May 19 12:25:31 2022] POST /identity/v3/projects => generated 355 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 12:25:31.363712 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a045bea-bce3-453d-a017-60be3c8b2da9 admin admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:25:31.374829 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a045bea-bce3-453d-a017-60be3c8b2da9 admin admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:25:31.436249 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1906783-a7a0-4ec5-889a-b2c8ef3f450e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:25:31.437122 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1906783-a7a0-4ec5-889a-b2c8ef3f450e admin admin] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:25:31.438782 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1906783-a7a0-4ec5-889a-b2c8ef3f450e admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:25:31.438782 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1906783-a7a0-4ec5-889a-b2c8ef3f450e admin admin] PATH_INFO: `/v3/users` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:25:31.438782 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1906783-a7a0-4ec5-889a-b2c8ef3f450e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:25:31.441235 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1906783-a7a0-4ec5-889a-b2c8ef3f450e admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:25:31.602941 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1184/2346] 10.222.0.23 () {60 vars in 1143 bytes} [Thu May 19 12:25:31 2022] POST /identity/v3/users => generated 278 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 12:25:31.606973 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42592a3f-2530-46a0-aad4-050563eb39a6 admin admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:25:31.619858 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42592a3f-2530-46a0-aad4-050563eb39a6 admin admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:25:31.625498 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9006c1-9c0c-4573-9e82-cb22c3442dd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:25:31.626260 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9006c1-9c0c-4573-9e82-cb22c3442dd4 admin admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:25:31.626709 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9006c1-9c0c-4573-9e82-cb22c3442dd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:25:31.627013 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9006c1-9c0c-4573-9e82-cb22c3442dd4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:25:31.628564 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9006c1-9c0c-4573-9e82-cb22c3442dd4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:25:31.630203 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9006c1-9c0c-4573-9e82-cb22c3442dd4 admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:25:31.642679 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1163/2347] 10.222.0.23 () {58 vars in 1121 bytes} [Thu May 19 12:25:31 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 19 12:25:31.647727 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1906783-a7a0-4ec5-889a-b2c8ef3f450e admin admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:25:31.657987 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1906783-a7a0-4ec5-889a-b2c8ef3f450e admin admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:25:31.663031 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d7abc4-c31f-4775-872c-83b0e6cd2570 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:25:31.663665 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d7abc4-c31f-4775-872c-83b0e6cd2570 admin admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:25:31.663984 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d7abc4-c31f-4775-872c-83b0e6cd2570 admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:25:31.664408 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d7abc4-c31f-4775-872c-83b0e6cd2570 admin admin] PATH_INFO: `/v3/roles` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:25:31.665165 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d7abc4-c31f-4775-872c-83b0e6cd2570 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:25:31.666787 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d7abc4-c31f-4775-872c-83b0e6cd2570 admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:25:31.677646 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1185/2348] 10.222.0.23 () {58 vars in 1121 bytes} [Thu May 19 12:25:31 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 19 12:25:31.681887 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9006c1-9c0c-4573-9e82-cb22c3442dd4 admin admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:25:31.692748 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9006c1-9c0c-4573-9e82-cb22c3442dd4 admin admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:25:31.697584 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63733223-fe92-49d7-bad8-68ce0d0b2152 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:25:31.698834 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63733223-fe92-49d7-bad8-68ce0d0b2152 admin admin] REQUEST_METHOD: `PUT` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:25:31.698940 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63733223-fe92-49d7-bad8-68ce0d0b2152 admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:25:31.699220 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63733223-fe92-49d7-bad8-68ce0d0b2152 admin admin] PATH_INFO: `/v3/projects/3e21979389b3414888d523c280ca62e5/users/2907849ae0cb4896bdf17a2c08a7f687/roles/259eb400cdb4439db72d14456ae9cd08` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:25:31.716350 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63733223-fe92-49d7-bad8-68ce0d0b2152 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3e21979389b3414888d523c280ca62e5, user_id=2907849ae0cb4896bdf17a2c08a7f687, role_id=259eb400cdb4439db72d14456ae9cd08)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:25:31.719358 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63733223-fe92-49d7-bad8-68ce0d0b2152 admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:25:31.802937 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1164/2349] 10.222.0.23 () {60 vars in 1482 bytes} [Thu May 19 12:25:31 2022] PUT /identity/v3/projects/3e21979389b3414888d523c280ca62e5/users/2907849ae0cb4896bdf17a2c08a7f687/roles/259eb400cdb4439db72d14456ae9cd08 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:25:31.814684 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f037d0-769f-4150-8835-6e9e002c9c4a None None] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:25:31.815871 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f037d0-769f-4150-8835-6e9e002c9c4a None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:25:31.816100 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f037d0-769f-4150-8835-6e9e002c9c4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:25:31.843498 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.auth.core [None req-a2f037d0-769f-4150-8835-6e9e002c9c4a None None] MFA Rules not processed for user `2907849ae0cb4896bdf17a2c08a7f687`. Rule list: `[]` (Enabled: `True`). {{(pid=80540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:25:31.844771 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-a2f037d0-769f-4150-8835-6e9e002c9c4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:25:31.913349 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1186/2350] 10.222.0.23 () {58 vars in 957 bytes} [Thu May 19 12:25:31 2022] POST /identity/v3/auth/tokens => generated 3222 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:25:31.917842 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63733223-fe92-49d7-bad8-68ce0d0b2152 admin admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:25:31.980243 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:25:31.995066 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63733223-fe92-49d7-bad8-68ce0d0b2152 admin admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:25:32.000634 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd9bf7d-8d19-4e98-bf62-f658d037db8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:25:32.001294 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd9bf7d-8d19-4e98-bf62-f658d037db8e admin admin] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:25:32.001583 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd9bf7d-8d19-4e98-bf62-f658d037db8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:25:32.001858 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd9bf7d-8d19-4e98-bf62-f658d037db8e admin admin] PATH_INFO: `/v3/projects` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:25:32.002825 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd9bf7d-8d19-4e98-bf62-f658d037db8e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:25:32.004917 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd9bf7d-8d19-4e98-bf62-f658d037db8e admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:25:32.137398 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1165/2351] 10.222.0.23 () {60 vars in 1152 bytes} [Thu May 19 12:25:31 2022] POST /identity/v3/projects => generated 355 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 12:25:32.141401 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f037d0-769f-4150-8835-6e9e002c9c4a None None] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:25:32.152725 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f037d0-769f-4150-8835-6e9e002c9c4a None None] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:25:32.156356 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:25:32.212708 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d331c1-592c-4195-b9a5-c90d15ab59ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:25:32.213687 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d331c1-592c-4195-b9a5-c90d15ab59ed admin admin] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:25:32.213980 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d331c1-592c-4195-b9a5-c90d15ab59ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:25:32.214265 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d331c1-592c-4195-b9a5-c90d15ab59ed admin admin] PATH_INFO: `/v3/users` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:25:32.215000 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d331c1-592c-4195-b9a5-c90d15ab59ed admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:25:32.217322 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d331c1-592c-4195-b9a5-c90d15ab59ed admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:25:32.371147 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1187/2352] 10.222.0.23 () {60 vars in 1143 bytes} [Thu May 19 12:25:32 2022] POST /identity/v3/users => generated 284 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 12:25:32.375121 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd9bf7d-8d19-4e98-bf62-f658d037db8e admin admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:25:32.386526 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd9bf7d-8d19-4e98-bf62-f658d037db8e admin admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:25:32.391321 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c7734d-f473-4980-9926-1f9d44253f99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:25:32.392006 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c7734d-f473-4980-9926-1f9d44253f99 admin admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:25:32.392308 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c7734d-f473-4980-9926-1f9d44253f99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:25:32.392618 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c7734d-f473-4980-9926-1f9d44253f99 admin admin] PATH_INFO: `/v3/roles` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:25:32.394021 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c7734d-f473-4980-9926-1f9d44253f99 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:25:32.395639 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c7734d-f473-4980-9926-1f9d44253f99 admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:25:32.406691 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1166/2353] 10.222.0.23 () {58 vars in 1121 bytes} [Thu May 19 12:25:32 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 12:25:32.412637 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d331c1-592c-4195-b9a5-c90d15ab59ed admin admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:25:32.417386 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801d57b6-3d7f-4805-8b90-f2579b8c7143 None None] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:25:32.417680 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801d57b6-3d7f-4805-8b90-f2579b8c7143 None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:25:32.417962 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801d57b6-3d7f-4805-8b90-f2579b8c7143 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:25:32.423677 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d331c1-592c-4195-b9a5-c90d15ab59ed admin admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:25:32.429500 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c784ae8-96be-4ef2-82ae-4e395fb11df7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:25:32.430401 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c784ae8-96be-4ef2-82ae-4e395fb11df7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:25:32.430693 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c784ae8-96be-4ef2-82ae-4e395fb11df7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:25:32.430983 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c784ae8-96be-4ef2-82ae-4e395fb11df7 admin admin] PATH_INFO: `/v3/projects/179aa7263d81461db754a422ac53e70c/users/74bd57ae16994d62bcc5fb9b60cc264e/roles/9db5671c65d3414ea2682afa2ccf1403` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:25:32.436265 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.auth.core [None req-801d57b6-3d7f-4805-8b90-f2579b8c7143 None None] MFA Rules not processed for user `3cdc4cc4d83b4495aa44de70147a001c`. Rule list: `[]` (Enabled: `True`). {{(pid=80539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:25:32.437525 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-801d57b6-3d7f-4805-8b90-f2579b8c7143 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:25:32.449693 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c784ae8-96be-4ef2-82ae-4e395fb11df7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=179aa7263d81461db754a422ac53e70c, user_id=74bd57ae16994d62bcc5fb9b60cc264e, role_id=9db5671c65d3414ea2682afa2ccf1403)` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:25:32.453246 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c784ae8-96be-4ef2-82ae-4e395fb11df7 admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:25:32.497496 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1167/2354] 10.222.0.23 () {58 vars in 957 bytes} [Thu May 19 12:25:32 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 12:25:32.639397 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1188/2355] 10.222.0.23 () {60 vars in 1482 bytes} [Thu May 19 12:25:32 2022] PUT /identity/v3/projects/179aa7263d81461db754a422ac53e70c/users/74bd57ae16994d62bcc5fb9b60cc264e/roles/9db5671c65d3414ea2682afa2ccf1403 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:25:32.643149 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801d57b6-3d7f-4805-8b90-f2579b8c7143 None None] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:25:32.655692 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801d57b6-3d7f-4805-8b90-f2579b8c7143 None None] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:25:32.715216 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92985583-5ca2-4146-bb99-cf92ccd36a7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:25:32.715918 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92985583-5ca2-4146-bb99-cf92ccd36a7b admin admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:25:32.716379 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92985583-5ca2-4146-bb99-cf92ccd36a7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:25:32.716535 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92985583-5ca2-4146-bb99-cf92ccd36a7b admin admin] PATH_INFO: `/v3/roles` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:25:32.717307 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92985583-5ca2-4146-bb99-cf92ccd36a7b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:25:32.718957 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92985583-5ca2-4146-bb99-cf92ccd36a7b admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:25:32.733200 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1168/2356] 10.222.0.23 () {58 vars in 1121 bytes} [Thu May 19 12:25:32 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 12:25:32.734900 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c784ae8-96be-4ef2-82ae-4e395fb11df7 admin admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:25:32.747183 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c784ae8-96be-4ef2-82ae-4e395fb11df7 admin admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:25:32.754091 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b79abd-e6d5-48d6-a3c1-ffc3a1ad934c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:25:32.754982 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b79abd-e6d5-48d6-a3c1-ffc3a1ad934c admin admin] REQUEST_METHOD: `PUT` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:25:32.755261 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b79abd-e6d5-48d6-a3c1-ffc3a1ad934c admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:25:32.756308 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b79abd-e6d5-48d6-a3c1-ffc3a1ad934c admin admin] PATH_INFO: `/v3/projects/179aa7263d81461db754a422ac53e70c/users/74bd57ae16994d62bcc5fb9b60cc264e/roles/9db5671c65d3414ea2682afa2ccf1403` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:25:32.757786 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b79abd-e6d5-48d6-a3c1-ffc3a1ad934c admin admin] RBAC: Authorizing `identity:create_grant(project_id=179aa7263d81461db754a422ac53e70c, user_id=74bd57ae16994d62bcc5fb9b60cc264e, role_id=9db5671c65d3414ea2682afa2ccf1403)` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:25:32.760523 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b79abd-e6d5-48d6-a3c1-ffc3a1ad934c admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:25:32.771678 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1189/2357] 10.222.0.23 () {60 vars in 1482 bytes} [Thu May 19 12:25:32 2022] PUT /identity/v3/projects/179aa7263d81461db754a422ac53e70c/users/74bd57ae16994d62bcc5fb9b60cc264e/roles/9db5671c65d3414ea2682afa2ccf1403 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:25:32.788191 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14da3bf-90f1-453e-a976-29a8b7fdf8d0 None None] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:25:32.788510 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14da3bf-90f1-453e-a976-29a8b7fdf8d0 None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:25:32.788815 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14da3bf-90f1-453e-a976-29a8b7fdf8d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:25:32.836943 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.auth.core [None req-c14da3bf-90f1-453e-a976-29a8b7fdf8d0 None None] MFA Rules not processed for user `74bd57ae16994d62bcc5fb9b60cc264e`. Rule list: `[]` (Enabled: `True`). {{(pid=80539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:25:32.838162 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-c14da3bf-90f1-453e-a976-29a8b7fdf8d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:25:32.894296 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:25:32.916490 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1169/2358] 10.222.0.23 () {58 vars in 957 bytes} [Thu May 19 12:25:32 2022] POST /identity/v3/auth/tokens => generated 3289 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:25:32.932492 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b79abd-e6d5-48d6-a3c1-ffc3a1ad934c admin admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:25:32.934482 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-b7b79abd-e6d5-48d6-a3c1-ffc3a1ad934c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:25:33.002792 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b79abd-e6d5-48d6-a3c1-ffc3a1ad934c admin admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:25:33.006768 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19855d6f-e791-4b55-9083-ba5db238fc97 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8800254c0e784e42a791fb9abc274d79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd70d25bf69314820b8aaecb65576978a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:25:33.007616 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19855d6f-e791-4b55-9083-ba5db238fc97 service cinder] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:25:33.008111 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19855d6f-e791-4b55-9083-ba5db238fc97 service cinder] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:25:33.008579 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19855d6f-e791-4b55-9083-ba5db238fc97 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:25:33.019141 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19855d6f-e791-4b55-9083-ba5db238fc97 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:25:33.020728 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19855d6f-e791-4b55-9083-ba5db238fc97 service cinder] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:25:33.030807 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1190/2359] 10.222.0.23 () {60 vars in 1365 bytes} [Thu May 19 12:25:32 2022] GET /identity/v3/auth/tokens => generated 3289 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:25:33.336869 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14da3bf-90f1-453e-a976-29a8b7fdf8d0 None None] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:25:33.349072 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14da3bf-90f1-453e-a976-29a8b7fdf8d0 None None] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:25:33.352842 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17e28fc-bc97-43e6-96f6-d02e308025cc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8800254c0e784e42a791fb9abc274d79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd70d25bf69314820b8aaecb65576978a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:25:33.354632 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17e28fc-bc97-43e6-96f6-d02e308025cc service cinder] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:25:33.354632 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17e28fc-bc97-43e6-96f6-d02e308025cc service cinder] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:25:33.354632 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17e28fc-bc97-43e6-96f6-d02e308025cc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:25:33.425893 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17e28fc-bc97-43e6-96f6-d02e308025cc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:25:33.427739 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17e28fc-bc97-43e6-96f6-d02e308025cc service cinder] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:25:33.439050 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1170/2360] 10.222.0.23 () {60 vars in 1365 bytes} [Thu May 19 12:25:33 2022] GET /identity/v3/auth/tokens => generated 3222 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:25:43.026290 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=67,keystone:INSERT=5 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:25:43.437052 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=34 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:25:43.486468 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f981fbb-bf8d-42a2-8bbe-4968ded414f7 None None] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:25:43.486757 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f981fbb-bf8d-42a2-8bbe-4968ded414f7 None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:25:43.487035 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f981fbb-bf8d-42a2-8bbe-4968ded414f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:25:43.505153 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.auth.core [None req-7f981fbb-bf8d-42a2-8bbe-4968ded414f7 None None] MFA Rules not processed for user `9d762cce00ae4e468c358b0437f35deb`. Rule list: `[]` (Enabled: `True`). {{(pid=80540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:25:43.506343 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-7f981fbb-bf8d-42a2-8bbe-4968ded414f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:25:43.561690 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1191/2361] 10.222.0.23 () {58 vars in 957 bytes} [Thu May 19 12:25:43 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:25:53.751882 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=25 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:25:56.584559 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17e28fc-bc97-43e6-96f6-d02e308025cc service cinder] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:25:56.651749 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17e28fc-bc97-43e6-96f6-d02e308025cc service cinder] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:25:56.656390 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d07f7d3-6aaf-45e4-81de-52832d23e510 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:25:56.657267 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d07f7d3-6aaf-45e4-81de-52832d23e510 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:25:56.657553 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d07f7d3-6aaf-45e4-81de-52832d23e510 admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:25:56.657886 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d07f7d3-6aaf-45e4-81de-52832d23e510 admin admin] PATH_INFO: `/v3/users/2907849ae0cb4896bdf17a2c08a7f687` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:25:56.659880 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d07f7d3-6aaf-45e4-81de-52832d23e510 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2907849ae0cb4896bdf17a2c08a7f687)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:25:56.661844 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d07f7d3-6aaf-45e4-81de-52832d23e510 admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:25:56.912727 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-0d07f7d3-6aaf-45e4-81de-52832d23e510 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2907849ae0cb4896bdf17a2c08a7f687'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:25:56.925698 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-0d07f7d3-6aaf-45e4-81de-52832d23e510 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2907849ae0cb4896bdf17a2c08a7f687'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:25:56.926532 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-0d07f7d3-6aaf-45e4-81de-52832d23e510 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2907849ae0cb4896bdf17a2c08a7f687'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:25:56.980499 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-0d07f7d3-6aaf-45e4-81de-52832d23e510 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2907849ae0cb4896bdf17a2c08a7f687'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:25:56.988981 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-0d07f7d3-6aaf-45e4-81de-52832d23e510 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2907849ae0cb4896bdf17a2c08a7f687'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:25:56.995393 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1171/2362] 10.222.0.23 () {58 vars in 1223 bytes} [Thu May 19 12:25:56 2022] DELETE /identity/v3/users/2907849ae0cb4896bdf17a2c08a7f687 => generated 0 bytes in 411 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:25:56.999956 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521c5fbf-5934-4fea-8dc3-31eb8953f5dd None None] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:25:57.000427 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521c5fbf-5934-4fea-8dc3-31eb8953f5dd None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:25:57.000533 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521c5fbf-5934-4fea-8dc3-31eb8953f5dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:25:57.020044 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.auth.core [None req-521c5fbf-5934-4fea-8dc3-31eb8953f5dd None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:25:57.021295 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-521c5fbf-5934-4fea-8dc3-31eb8953f5dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:25:57.082507 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1192/2363] 10.222.0.23 () {58 vars in 957 bytes} [Thu May 19 12:25:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:25:57.092271 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d07f7d3-6aaf-45e4-81de-52832d23e510 admin admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:25:57.093672 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-0d07f7d3-6aaf-45e4-81de-52832d23e510 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:25:57.151806 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d07f7d3-6aaf-45e4-81de-52832d23e510 admin admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:25:57.155384 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab943948-177e-4ca2-8c62-c68a7d028824 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a00b3fa7497456895e5e0bf105c4fc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd70d25bf69314820b8aaecb65576978a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:25:57.156048 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab943948-177e-4ca2-8c62-c68a7d028824 service neutron] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:25:57.156330 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab943948-177e-4ca2-8c62-c68a7d028824 service neutron] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:25:57.156602 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab943948-177e-4ca2-8c62-c68a7d028824 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:25:57.165992 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab943948-177e-4ca2-8c62-c68a7d028824 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:25:57.167413 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab943948-177e-4ca2-8c62-c68a7d028824 service neutron] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:25:57.178065 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1172/2364] 10.222.0.23 () {60 vars in 1365 bytes} [Thu May 19 12:25:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:25:57.645196 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521c5fbf-5934-4fea-8dc3-31eb8953f5dd None None] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:25:57.649044 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-521c5fbf-5934-4fea-8dc3-31eb8953f5dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:25:57.661190 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521c5fbf-5934-4fea-8dc3-31eb8953f5dd None None] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:25:57.665989 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328fd2d1-62d6-43f0-8bcf-946de1cc8e1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:25:57.666903 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328fd2d1-62d6-43f0-8bcf-946de1cc8e1c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:25:57.667193 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328fd2d1-62d6-43f0-8bcf-946de1cc8e1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:25:57.671816 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328fd2d1-62d6-43f0-8bcf-946de1cc8e1c admin admin] PATH_INFO: `/v3/projects/3e21979389b3414888d523c280ca62e5` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:25:57.671816 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328fd2d1-62d6-43f0-8bcf-946de1cc8e1c admin admin] RBAC: Authorizing `identity:delete_project(project_id=3e21979389b3414888d523c280ca62e5)` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:25:57.671816 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328fd2d1-62d6-43f0-8bcf-946de1cc8e1c admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:25:57.774974 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-328fd2d1-62d6-43f0-8bcf-946de1cc8e1c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3e21979389b3414888d523c280ca62e5', 'request_id': 'req-328fd2d1-62d6-43f0-8bcf-946de1cc8e1c'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:25:57.781098 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-328fd2d1-62d6-43f0-8bcf-946de1cc8e1c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3e21979389b3414888d523c280ca62e5', 'request_id': 'req-328fd2d1-62d6-43f0-8bcf-946de1cc8e1c'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:25:57.788069 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-328fd2d1-62d6-43f0-8bcf-946de1cc8e1c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3e21979389b3414888d523c280ca62e5', 'request_id': 'req-328fd2d1-62d6-43f0-8bcf-946de1cc8e1c'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:25:57.881089 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-328fd2d1-62d6-43f0-8bcf-946de1cc8e1c admin admin] The token cache is being invalidate because project 3e21979389b3414888d523c280ca62e5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80540) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 12:25:57.881382 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-328fd2d1-62d6-43f0-8bcf-946de1cc8e1c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:25:57.882017 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-328fd2d1-62d6-43f0-8bcf-946de1cc8e1c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:25:57.883570 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1193/2365] 10.222.0.23 () {58 vars in 1232 bytes} [Thu May 19 12:25:57 2022] DELETE /identity/v3/projects/3e21979389b3414888d523c280ca62e5 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:25:57.887631 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab943948-177e-4ca2-8c62-c68a7d028824 service neutron] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:25:57.898789 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab943948-177e-4ca2-8c62-c68a7d028824 service neutron] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:25:57.917316 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:25:57.957665 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d169143-2dcc-4a73-a5da-fa1a5a992a81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:25:57.958551 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d169143-2dcc-4a73-a5da-fa1a5a992a81 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:25:57.958866 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d169143-2dcc-4a73-a5da-fa1a5a992a81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:25:57.959145 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d169143-2dcc-4a73-a5da-fa1a5a992a81 admin admin] PATH_INFO: `/v3/users/74bd57ae16994d62bcc5fb9b60cc264e` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:25:57.960410 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d169143-2dcc-4a73-a5da-fa1a5a992a81 admin admin] RBAC: Authorizing `identity:delete_user(user_id=74bd57ae16994d62bcc5fb9b60cc264e)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:25:57.962384 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d169143-2dcc-4a73-a5da-fa1a5a992a81 admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:25:58.188522 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-0d169143-2dcc-4a73-a5da-fa1a5a992a81 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '74bd57ae16994d62bcc5fb9b60cc264e'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:25:58.200720 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-0d169143-2dcc-4a73-a5da-fa1a5a992a81 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '74bd57ae16994d62bcc5fb9b60cc264e'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:25:58.201424 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-0d169143-2dcc-4a73-a5da-fa1a5a992a81 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '74bd57ae16994d62bcc5fb9b60cc264e'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:25:58.280865 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-0d169143-2dcc-4a73-a5da-fa1a5a992a81 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '74bd57ae16994d62bcc5fb9b60cc264e'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:25:58.288742 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-0d169143-2dcc-4a73-a5da-fa1a5a992a81 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '74bd57ae16994d62bcc5fb9b60cc264e'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:25:58.294727 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1173/2366] 10.222.0.23 () {58 vars in 1223 bytes} [Thu May 19 12:25:57 2022] DELETE /identity/v3/users/74bd57ae16994d62bcc5fb9b60cc264e => generated 0 bytes in 408 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:25:58.756432 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328fd2d1-62d6-43f0-8bcf-946de1cc8e1c admin admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:25:58.757900 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-328fd2d1-62d6-43f0-8bcf-946de1cc8e1c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:25:58.771895 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328fd2d1-62d6-43f0-8bcf-946de1cc8e1c admin admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:25:58.837433 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9d26fe-42ae-48fb-9c4c-4a656ea6b5fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:25:58.838306 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9d26fe-42ae-48fb-9c4c-4a656ea6b5fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:25:58.838592 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9d26fe-42ae-48fb-9c4c-4a656ea6b5fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:25:58.838918 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9d26fe-42ae-48fb-9c4c-4a656ea6b5fa admin admin] PATH_INFO: `/v3/projects/179aa7263d81461db754a422ac53e70c` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:25:58.840143 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9d26fe-42ae-48fb-9c4c-4a656ea6b5fa admin admin] RBAC: Authorizing `identity:delete_project(project_id=179aa7263d81461db754a422ac53e70c)` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:25:58.842090 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9d26fe-42ae-48fb-9c4c-4a656ea6b5fa admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:25:58.962267 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:25:58.980723 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-4f9d26fe-42ae-48fb-9c4c-4a656ea6b5fa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '179aa7263d81461db754a422ac53e70c', 'request_id': 'req-4f9d26fe-42ae-48fb-9c4c-4a656ea6b5fa'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:25:58.987964 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-4f9d26fe-42ae-48fb-9c4c-4a656ea6b5fa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '179aa7263d81461db754a422ac53e70c', 'request_id': 'req-4f9d26fe-42ae-48fb-9c4c-4a656ea6b5fa'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:25:58.993594 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-4f9d26fe-42ae-48fb-9c4c-4a656ea6b5fa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '179aa7263d81461db754a422ac53e70c', 'request_id': 'req-4f9d26fe-42ae-48fb-9c4c-4a656ea6b5fa'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:25:59.123069 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-4f9d26fe-42ae-48fb-9c4c-4a656ea6b5fa admin admin] The token cache is being invalidate because project 179aa7263d81461db754a422ac53e70c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80540) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 12:25:59.123380 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-4f9d26fe-42ae-48fb-9c4c-4a656ea6b5fa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:25:59.124069 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-4f9d26fe-42ae-48fb-9c4c-4a656ea6b5fa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:25:59.125671 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1194/2367] 10.222.0.23 () {58 vars in 1232 bytes} [Thu May 19 12:25:58 2022] DELETE /identity/v3/projects/179aa7263d81461db754a422ac53e70c => generated 0 bytes in 370 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:25:59.134297 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad87cc42-7957-4c5d-a72d-dad5bd2c209f None None] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:25:59.134297 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad87cc42-7957-4c5d-a72d-dad5bd2c209f None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:25:59.134297 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad87cc42-7957-4c5d-a72d-dad5bd2c209f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:25:59.150908 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.auth.core [None req-ad87cc42-7957-4c5d-a72d-dad5bd2c209f None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:25:59.152182 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-ad87cc42-7957-4c5d-a72d-dad5bd2c209f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:25:59.214122 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1174/2368] 10.222.0.23 () {58 vars in 957 bytes} [Thu May 19 12:25:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:25:59.224080 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa47eec-6c05-43f5-b322-7ba5c558d358 None None] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:25:59.224374 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa47eec-6c05-43f5-b322-7ba5c558d358 None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:25:59.224656 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa47eec-6c05-43f5-b322-7ba5c558d358 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:25:59.243992 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.auth.core [None req-6fa47eec-6c05-43f5-b322-7ba5c558d358 None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:25:59.245209 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-6fa47eec-6c05-43f5-b322-7ba5c558d358 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:25:59.253239 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1195/2369] 10.222.0.23 () {58 vars in 957 bytes} [Thu May 19 12:25:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:25:59.257506 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad87cc42-7957-4c5d-a72d-dad5bd2c209f None None] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:25:59.267376 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad87cc42-7957-4c5d-a72d-dad5bd2c209f None None] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:25:59.272217 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5d14fc-3cb1-43d0-b174-bd8dc28b0313 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:25:59.272873 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5d14fc-3cb1-43d0-b174-bd8dc28b0313 admin admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:25:59.273185 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5d14fc-3cb1-43d0-b174-bd8dc28b0313 admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:25:59.273464 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5d14fc-3cb1-43d0-b174-bd8dc28b0313 admin admin] PATH_INFO: `/v3/domains` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:25:59.274196 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5d14fc-3cb1-43d0-b174-bd8dc28b0313 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:25:59.274506 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5d14fc-3cb1-43d0-b174-bd8dc28b0313 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:25:59.276134 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5d14fc-3cb1-43d0-b174-bd8dc28b0313 admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:25:59.288823 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1175/2370] 10.222.0.23 () {58 vars in 1152 bytes} [Thu May 19 12:25:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:25:59.293229 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa47eec-6c05-43f5-b322-7ba5c558d358 None None] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:25:59.304633 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa47eec-6c05-43f5-b322-7ba5c558d358 None None] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:25:59.309366 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc0a147-e790-41e9-ab6b-0ff0eae7b4a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:25:59.310017 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc0a147-e790-41e9-ab6b-0ff0eae7b4a4 admin admin] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:25:59.312134 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc0a147-e790-41e9-ab6b-0ff0eae7b4a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:25:59.312134 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc0a147-e790-41e9-ab6b-0ff0eae7b4a4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:25:59.312134 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc0a147-e790-41e9-ab6b-0ff0eae7b4a4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:25:59.313258 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc0a147-e790-41e9-ab6b-0ff0eae7b4a4 admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:25:59.394132 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1196/2371] 10.222.0.23 () {60 vars in 1152 bytes} [Thu May 19 12:25:59 2022] POST /identity/v3/projects => generated 375 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 12:25:59.398621 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5d14fc-3cb1-43d0-b174-bd8dc28b0313 admin admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:25:59.409414 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5d14fc-3cb1-43d0-b174-bd8dc28b0313 admin admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:25:59.436149 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:25:59.466225 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e3fc9f-8bcb-41f1-8427-31bdf452b572 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:25:59.466905 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e3fc9f-8bcb-41f1-8427-31bdf452b572 admin admin] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:25:59.467192 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e3fc9f-8bcb-41f1-8427-31bdf452b572 admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:25:59.467470 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e3fc9f-8bcb-41f1-8427-31bdf452b572 admin admin] PATH_INFO: `/v3/users` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:25:59.468203 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e3fc9f-8bcb-41f1-8427-31bdf452b572 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:25:59.470155 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e3fc9f-8bcb-41f1-8427-31bdf452b572 admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:25:59.581529 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1176/2372] 10.222.0.23 () {60 vars in 1143 bytes} [Thu May 19 12:25:59 2022] POST /identity/v3/users => generated 288 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 12:25:59.585796 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc0a147-e790-41e9-ab6b-0ff0eae7b4a4 admin admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:25:59.597249 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc0a147-e790-41e9-ab6b-0ff0eae7b4a4 admin admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:25:59.602436 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d3e543-684c-412c-9dc3-6430f0290d75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:25:59.604526 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d3e543-684c-412c-9dc3-6430f0290d75 admin admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:25:59.604526 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d3e543-684c-412c-9dc3-6430f0290d75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:25:59.604526 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d3e543-684c-412c-9dc3-6430f0290d75 admin admin] PATH_INFO: `/v3/roles` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:25:59.605334 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d3e543-684c-412c-9dc3-6430f0290d75 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:25:59.606195 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d3e543-684c-412c-9dc3-6430f0290d75 admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:25:59.618124 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1197/2373] 10.222.0.23 () {58 vars in 1121 bytes} [Thu May 19 12:25:59 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 12:25:59.622273 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e3fc9f-8bcb-41f1-8427-31bdf452b572 admin admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:25:59.633085 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e3fc9f-8bcb-41f1-8427-31bdf452b572 admin admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:25:59.637926 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b243af1b-8c5f-457c-9e3f-f43992d79a2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:25:59.638596 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b243af1b-8c5f-457c-9e3f-f43992d79a2a admin admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:25:59.638880 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b243af1b-8c5f-457c-9e3f-f43992d79a2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:25:59.639163 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b243af1b-8c5f-457c-9e3f-f43992d79a2a admin admin] PATH_INFO: `/v3/roles` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:25:59.639994 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b243af1b-8c5f-457c-9e3f-f43992d79a2a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:25:59.641638 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b243af1b-8c5f-457c-9e3f-f43992d79a2a admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:25:59.658456 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1177/2374] 10.222.0.23 () {58 vars in 1121 bytes} [Thu May 19 12:25:59 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 12:25:59.661593 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d3e543-684c-412c-9dc3-6430f0290d75 admin admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:25:59.672454 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d3e543-684c-412c-9dc3-6430f0290d75 admin admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:25:59.677997 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd9baaa-86b4-4e5c-b2b4-09ee86c540ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:25:59.678901 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd9baaa-86b4-4e5c-b2b4-09ee86c540ce admin admin] REQUEST_METHOD: `PUT` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:25:59.679209 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd9baaa-86b4-4e5c-b2b4-09ee86c540ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:25:59.679491 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd9baaa-86b4-4e5c-b2b4-09ee86c540ce admin admin] PATH_INFO: `/v3/projects/e3b28e8808774b989c36a5c3ac34dda0/users/22f71727fe7e45e9a5601382d8f8e306/roles/259eb400cdb4439db72d14456ae9cd08` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:25:59.695509 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd9baaa-86b4-4e5c-b2b4-09ee86c540ce admin admin] RBAC: Authorizing `identity:create_grant(project_id=e3b28e8808774b989c36a5c3ac34dda0, user_id=22f71727fe7e45e9a5601382d8f8e306, role_id=259eb400cdb4439db72d14456ae9cd08)` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:25:59.698338 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd9baaa-86b4-4e5c-b2b4-09ee86c540ce admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:25:59.775754 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1198/2375] 10.222.0.23 () {60 vars in 1482 bytes} [Thu May 19 12:25:59 2022] PUT /identity/v3/projects/e3b28e8808774b989c36a5c3ac34dda0/users/22f71727fe7e45e9a5601382d8f8e306/roles/259eb400cdb4439db72d14456ae9cd08 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:25:59.785887 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e658b40-c7e7-47cb-8fdd-5cdf8d434a7f None None] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:25:59.786847 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e658b40-c7e7-47cb-8fdd-5cdf8d434a7f None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:25:59.787163 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e658b40-c7e7-47cb-8fdd-5cdf8d434a7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:25:59.816933 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.auth.core [None req-3e658b40-c7e7-47cb-8fdd-5cdf8d434a7f None None] MFA Rules not processed for user `22f71727fe7e45e9a5601382d8f8e306`. Rule list: `[]` (Enabled: `True`). {{(pid=80539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:25:59.818178 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-3e658b40-c7e7-47cb-8fdd-5cdf8d434a7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:25:59.890482 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1178/2376] 10.222.0.23 () {58 vars in 957 bytes} [Thu May 19 12:25:59 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:25:59.902186 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd9baaa-86b4-4e5c-b2b4-09ee86c540ce admin admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:25:59.903903 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-fdd9baaa-86b4-4e5c-b2b4-09ee86c540ce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:25:59.965770 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd9baaa-86b4-4e5c-b2b4-09ee86c540ce admin admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:25:59.970070 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277ddf8e-ed82-486e-9c21-b57e2cfa68ba service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8800254c0e784e42a791fb9abc274d79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd70d25bf69314820b8aaecb65576978a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:25:59.970847 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277ddf8e-ed82-486e-9c21-b57e2cfa68ba service cinder] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:25:59.971133 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277ddf8e-ed82-486e-9c21-b57e2cfa68ba service cinder] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:25:59.971408 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277ddf8e-ed82-486e-9c21-b57e2cfa68ba service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:25:59.981280 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277ddf8e-ed82-486e-9c21-b57e2cfa68ba service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:25:59.982723 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277ddf8e-ed82-486e-9c21-b57e2cfa68ba service cinder] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:25:59.992458 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1199/2377] 10.222.0.23 () {60 vars in 1365 bytes} [Thu May 19 12:25:59 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:26:09.887620 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=53,keystone:INSERT=3 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:26:09.988709 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:DELETE=6,keystone:SELECT=67,keystone:INSERT=3 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:26:22.830979 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dedef5b-a986-4ad6-a4e6-17ae79471f78 None None] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:26:22.831899 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dedef5b-a986-4ad6-a4e6-17ae79471f78 None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:26:22.832197 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dedef5b-a986-4ad6-a4e6-17ae79471f78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:26:22.850614 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.auth.core [None req-4dedef5b-a986-4ad6-a4e6-17ae79471f78 None None] MFA Rules not processed for user `0c3181fb4f584a549a786adfefdf3dfb`. Rule list: `[]` (Enabled: `True`). {{(pid=80539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:26:22.851905 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-4dedef5b-a986-4ad6-a4e6-17ae79471f78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:26:22.908909 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1179/2378] 10.222.0.23 () {58 vars in 967 bytes} [Thu May 19 12:26:22 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:26:23.056444 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277ddf8e-ed82-486e-9c21-b57e2cfa68ba service cinder] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:26:23.057862 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-277ddf8e-ed82-486e-9c21-b57e2cfa68ba service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:26:23.127621 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277ddf8e-ed82-486e-9c21-b57e2cfa68ba service cinder] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:26:23.132818 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b011e2b-9f25-40ae-962b-17a415f045a8 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15cbfc73ae1243d2a8b1e70f9cd99876', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd70d25bf69314820b8aaecb65576978a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:26:23.133474 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b011e2b-9f25-40ae-962b-17a415f045a8 service swift] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:26:23.133764 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b011e2b-9f25-40ae-962b-17a415f045a8 service swift] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:26:23.134049 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b011e2b-9f25-40ae-962b-17a415f045a8 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:26:23.142380 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b011e2b-9f25-40ae-962b-17a415f045a8 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:26:23.144087 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b011e2b-9f25-40ae-962b-17a415f045a8 service swift] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:26:23.152006 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1200/2379] 10.222.0.23 () {60 vars in 1384 bytes} [Thu May 19 12:26:23 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 19 12:26:25.863776 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dedef5b-a986-4ad6-a4e6-17ae79471f78 None None] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:26:25.865530 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-4dedef5b-a986-4ad6-a4e6-17ae79471f78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:26:25.936241 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dedef5b-a986-4ad6-a4e6-17ae79471f78 None None] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:26:25.940177 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec8da96-85b8-417c-9a47-84a2e03757f9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a00b3fa7497456895e5e0bf105c4fc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd70d25bf69314820b8aaecb65576978a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:26:25.941576 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec8da96-85b8-417c-9a47-84a2e03757f9 service neutron] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:26:25.941931 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec8da96-85b8-417c-9a47-84a2e03757f9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:26:25.942310 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec8da96-85b8-417c-9a47-84a2e03757f9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:26:25.944299 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-6ec8da96-85b8-417c-9a47-84a2e03757f9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:26:26.041473 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec8da96-85b8-417c-9a47-84a2e03757f9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:26:26.043054 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec8da96-85b8-417c-9a47-84a2e03757f9 service neutron] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:26:26.054993 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1180/2380] 10.222.0.23 () {60 vars in 1365 bytes} [Thu May 19 12:26:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:26:32.952334 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b011e2b-9f25-40ae-962b-17a415f045a8 service swift] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:26:32.954018 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-9b011e2b-9f25-40ae-962b-17a415f045a8 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:26:32.968938 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b011e2b-9f25-40ae-962b-17a415f045a8 service swift] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:26:32.973841 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4dab5f-051c-41f0-a287-d75c149e6945 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:26:32.974712 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4dab5f-051c-41f0-a287-d75c149e6945 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:26:32.974998 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4dab5f-051c-41f0-a287-d75c149e6945 admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:26:32.975305 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4dab5f-051c-41f0-a287-d75c149e6945 admin admin] PATH_INFO: `/v3/users/d53ee7f4a9b74ce1868715ed0177bf2a` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:26:32.976591 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4dab5f-051c-41f0-a287-d75c149e6945 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d53ee7f4a9b74ce1868715ed0177bf2a)` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:26:32.978529 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4dab5f-051c-41f0-a287-d75c149e6945 admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:26:33.250891 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-be4dab5f-051c-41f0-a287-d75c149e6945 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd53ee7f4a9b74ce1868715ed0177bf2a'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:26:33.263443 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-be4dab5f-051c-41f0-a287-d75c149e6945 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd53ee7f4a9b74ce1868715ed0177bf2a'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:26:33.264153 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-be4dab5f-051c-41f0-a287-d75c149e6945 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd53ee7f4a9b74ce1868715ed0177bf2a'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:26:33.339228 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-be4dab5f-051c-41f0-a287-d75c149e6945 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd53ee7f4a9b74ce1868715ed0177bf2a'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:26:33.347073 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-be4dab5f-051c-41f0-a287-d75c149e6945 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd53ee7f4a9b74ce1868715ed0177bf2a'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:26:33.353218 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1201/2381] 10.222.0.23 () {58 vars in 1223 bytes} [Thu May 19 12:26:32 2022] DELETE /identity/v3/users/d53ee7f4a9b74ce1868715ed0177bf2a => generated 0 bytes in 402 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:26:33.621190 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec8da96-85b8-417c-9a47-84a2e03757f9 service neutron] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:26:33.632485 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec8da96-85b8-417c-9a47-84a2e03757f9 service neutron] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:26:33.694583 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:26:33.701876 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e725c5-5c17-439b-b42e-3d1c72709a29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:26:33.703554 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e725c5-5c17-439b-b42e-3d1c72709a29 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:26:33.703554 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e725c5-5c17-439b-b42e-3d1c72709a29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:26:33.703554 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e725c5-5c17-439b-b42e-3d1c72709a29 admin admin] PATH_INFO: `/v3/projects/37c1366ee71c446b9ec6fa532391b789` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:26:33.704624 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e725c5-5c17-439b-b42e-3d1c72709a29 admin admin] RBAC: Authorizing `identity:delete_project(project_id=37c1366ee71c446b9ec6fa532391b789)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:26:33.706613 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e725c5-5c17-439b-b42e-3d1c72709a29 admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:26:33.882123 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-15e725c5-5c17-439b-b42e-3d1c72709a29 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '37c1366ee71c446b9ec6fa532391b789', 'request_id': 'req-15e725c5-5c17-439b-b42e-3d1c72709a29'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:26:33.888474 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-15e725c5-5c17-439b-b42e-3d1c72709a29 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '37c1366ee71c446b9ec6fa532391b789', 'request_id': 'req-15e725c5-5c17-439b-b42e-3d1c72709a29'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:26:33.927349 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4dab5f-051c-41f0-a287-d75c149e6945 admin admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:26:33.928943 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-be4dab5f-051c-41f0-a287-d75c149e6945 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:26:33.942695 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4dab5f-051c-41f0-a287-d75c149e6945 admin admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:26:34.002260 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83624d04-e06a-43eb-88fd-600a1db7e1cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:26:34.003374 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83624d04-e06a-43eb-88fd-600a1db7e1cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:26:34.003662 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83624d04-e06a-43eb-88fd-600a1db7e1cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:26:34.004184 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83624d04-e06a-43eb-88fd-600a1db7e1cf admin admin] PATH_INFO: `/v3/users/22f71727fe7e45e9a5601382d8f8e306` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:26:34.005393 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83624d04-e06a-43eb-88fd-600a1db7e1cf admin admin] RBAC: Authorizing `identity:delete_user(user_id=22f71727fe7e45e9a5601382d8f8e306)` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:26:34.007333 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83624d04-e06a-43eb-88fd-600a1db7e1cf admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:26:34.009833 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:26:34.051343 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-15e725c5-5c17-439b-b42e-3d1c72709a29 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '37c1366ee71c446b9ec6fa532391b789', 'request_id': 'req-15e725c5-5c17-439b-b42e-3d1c72709a29'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:26:34.062608 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-15e725c5-5c17-439b-b42e-3d1c72709a29 admin admin] The token cache is being invalidate because project 37c1366ee71c446b9ec6fa532391b789 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80539) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 12:26:34.062952 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-15e725c5-5c17-439b-b42e-3d1c72709a29 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:26:34.063620 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-15e725c5-5c17-439b-b42e-3d1c72709a29 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:26:34.065292 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1181/2382] 10.222.0.23 () {58 vars in 1232 bytes} [Thu May 19 12:26:33 2022] DELETE /identity/v3/projects/37c1366ee71c446b9ec6fa532391b789 => generated 0 bytes in 445 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:26:34.069335 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e725c5-5c17-439b-b42e-3d1c72709a29 admin admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:26:34.070814 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-15e725c5-5c17-439b-b42e-3d1c72709a29 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:26:34.084787 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e725c5-5c17-439b-b42e-3d1c72709a29 admin admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:26:34.089804 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a03525-f77f-4091-bcf2-26f85a07351e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:26:34.090663 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a03525-f77f-4091-bcf2-26f85a07351e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:26:34.090975 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a03525-f77f-4091-bcf2-26f85a07351e admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:26:34.091252 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a03525-f77f-4091-bcf2-26f85a07351e admin admin] PATH_INFO: `/v3/users/ef2523c3639448cf8697032a07e79fb4` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:26:34.092561 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a03525-f77f-4091-bcf2-26f85a07351e admin admin] RBAC: Authorizing `identity:delete_user(user_id=ef2523c3639448cf8697032a07e79fb4)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:26:34.094515 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a03525-f77f-4091-bcf2-26f85a07351e admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:26:34.255890 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-83624d04-e06a-43eb-88fd-600a1db7e1cf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '22f71727fe7e45e9a5601382d8f8e306'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:26:34.267365 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-83624d04-e06a-43eb-88fd-600a1db7e1cf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '22f71727fe7e45e9a5601382d8f8e306'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:26:34.268077 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-83624d04-e06a-43eb-88fd-600a1db7e1cf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '22f71727fe7e45e9a5601382d8f8e306'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:26:34.305535 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-55a03525-f77f-4091-bcf2-26f85a07351e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ef2523c3639448cf8697032a07e79fb4'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:26:34.318662 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-55a03525-f77f-4091-bcf2-26f85a07351e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ef2523c3639448cf8697032a07e79fb4'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:26:34.319357 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-55a03525-f77f-4091-bcf2-26f85a07351e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ef2523c3639448cf8697032a07e79fb4'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:26:34.379852 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-83624d04-e06a-43eb-88fd-600a1db7e1cf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '22f71727fe7e45e9a5601382d8f8e306'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:26:34.387419 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-83624d04-e06a-43eb-88fd-600a1db7e1cf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '22f71727fe7e45e9a5601382d8f8e306'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:26:34.394777 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1202/2383] 10.222.0.23 () {58 vars in 1223 bytes} [Thu May 19 12:26:33 2022] DELETE /identity/v3/users/22f71727fe7e45e9a5601382d8f8e306 => generated 0 bytes in 468 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 12:26:34.399255 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce55a6bf-6178-4b69-b1f5-5b66a2fd5937 None None] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:26:34.399642 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce55a6bf-6178-4b69-b1f5-5b66a2fd5937 None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:26:34.400080 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce55a6bf-6178-4b69-b1f5-5b66a2fd5937 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:26:34.419106 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.auth.core [None req-ce55a6bf-6178-4b69-b1f5-5b66a2fd5937 None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:26:34.420772 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-ce55a6bf-6178-4b69-b1f5-5b66a2fd5937 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:26:34.480121 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1203/2384] 10.222.0.23 () {58 vars in 957 bytes} [Thu May 19 12:26:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:26:34.490409 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce55a6bf-6178-4b69-b1f5-5b66a2fd5937 None None] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:26:34.491188 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-55a03525-f77f-4091-bcf2-26f85a07351e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ef2523c3639448cf8697032a07e79fb4'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:26:34.491987 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-ce55a6bf-6178-4b69-b1f5-5b66a2fd5937 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:26:34.501205 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-55a03525-f77f-4091-bcf2-26f85a07351e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ef2523c3639448cf8697032a07e79fb4'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:26:34.509277 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1182/2385] 10.222.0.23 () {58 vars in 1223 bytes} [Thu May 19 12:26:34 2022] DELETE /identity/v3/users/ef2523c3639448cf8697032a07e79fb4 => generated 0 bytes in 439 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 12:26:34.552556 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce55a6bf-6178-4b69-b1f5-5b66a2fd5937 None None] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:26:34.590352 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:26:34.607056 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2a2664-00d3-41e7-a397-aaf2b7a13576 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a00b3fa7497456895e5e0bf105c4fc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd70d25bf69314820b8aaecb65576978a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:26:34.607839 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2a2664-00d3-41e7-a397-aaf2b7a13576 service neutron] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:26:34.608142 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2a2664-00d3-41e7-a397-aaf2b7a13576 service neutron] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:26:34.608427 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2a2664-00d3-41e7-a397-aaf2b7a13576 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:26:34.610853 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-7a2a2664-00d3-41e7-a397-aaf2b7a13576 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:26:34.677044 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2a2664-00d3-41e7-a397-aaf2b7a13576 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:26:34.678514 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2a2664-00d3-41e7-a397-aaf2b7a13576 service neutron] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:26:34.688220 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1204/2386] 10.222.0.23 () {60 vars in 1365 bytes} [Thu May 19 12:26:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 198 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 12:26:34.853380 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a03525-f77f-4091-bcf2-26f85a07351e admin admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:26:34.854847 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-55a03525-f77f-4091-bcf2-26f85a07351e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:26:34.870057 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a03525-f77f-4091-bcf2-26f85a07351e admin admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:26:34.876533 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b055e0-33a4-492f-8f8a-0f720b5e4e37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:26:34.877409 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b055e0-33a4-492f-8f8a-0f720b5e4e37 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:26:34.879066 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b055e0-33a4-492f-8f8a-0f720b5e4e37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:26:34.879066 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b055e0-33a4-492f-8f8a-0f720b5e4e37 admin admin] PATH_INFO: `/v3/projects/22ac51f7867c4e50a7ee4863f7f7aa28` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:26:34.879898 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b055e0-33a4-492f-8f8a-0f720b5e4e37 admin admin] RBAC: Authorizing `identity:delete_project(project_id=22ac51f7867c4e50a7ee4863f7f7aa28)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:26:34.881898 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b055e0-33a4-492f-8f8a-0f720b5e4e37 admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:26:35.049153 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-a2b055e0-33a4-492f-8f8a-0f720b5e4e37 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '22ac51f7867c4e50a7ee4863f7f7aa28', 'request_id': 'req-a2b055e0-33a4-492f-8f8a-0f720b5e4e37'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:26:35.051974 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=77,keystone:DELETE=21,keystone:INSERT=2 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:26:35.056490 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-a2b055e0-33a4-492f-8f8a-0f720b5e4e37 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '22ac51f7867c4e50a7ee4863f7f7aa28', 'request_id': 'req-a2b055e0-33a4-492f-8f8a-0f720b5e4e37'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:26:35.167993 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-a2b055e0-33a4-492f-8f8a-0f720b5e4e37 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '22ac51f7867c4e50a7ee4863f7f7aa28', 'request_id': 'req-a2b055e0-33a4-492f-8f8a-0f720b5e4e37'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:26:35.180256 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-a2b055e0-33a4-492f-8f8a-0f720b5e4e37 admin admin] The token cache is being invalidate because project 22ac51f7867c4e50a7ee4863f7f7aa28 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80539) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 12:26:35.180545 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-a2b055e0-33a4-492f-8f8a-0f720b5e4e37 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:26:35.181197 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-a2b055e0-33a4-492f-8f8a-0f720b5e4e37 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:26:35.182776 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1183/2387] 10.222.0.23 () {58 vars in 1232 bytes} [Thu May 19 12:26:34 2022] DELETE /identity/v3/projects/22ac51f7867c4e50a7ee4863f7f7aa28 => generated 0 bytes in 330 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:26:35.187800 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b364a690-7239-4aed-8a9f-c7d338aa0d02 None None] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:26:35.188263 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b364a690-7239-4aed-8a9f-c7d338aa0d02 None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:26:35.188451 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b364a690-7239-4aed-8a9f-c7d338aa0d02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:26:35.207415 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.auth.core [None req-b364a690-7239-4aed-8a9f-c7d338aa0d02 None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:26:35.208173 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b055e0-33a4-492f-8f8a-0f720b5e4e37 admin admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:26:35.208545 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-b364a690-7239-4aed-8a9f-c7d338aa0d02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:26:35.211226 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-a2b055e0-33a4-492f-8f8a-0f720b5e4e37 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:26:35.277823 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1205/2388] 10.222.0.23 () {58 vars in 957 bytes} [Thu May 19 12:26:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:26:35.281481 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b055e0-33a4-492f-8f8a-0f720b5e4e37 admin admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:26:35.286781 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14e3826-efb0-45c6-b49c-545210e148ce None None] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:26:35.287080 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14e3826-efb0-45c6-b49c-545210e148ce None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:26:35.287208 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab4120d-1345-45bb-b124-14b73465d232 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:26:35.287388 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14e3826-efb0-45c6-b49c-545210e148ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:26:35.288030 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab4120d-1345-45bb-b124-14b73465d232 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:26:35.288258 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab4120d-1345-45bb-b124-14b73465d232 admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:26:35.288536 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab4120d-1345-45bb-b124-14b73465d232 admin admin] PATH_INFO: `/v3/projects/e3b28e8808774b989c36a5c3ac34dda0` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:26:35.289709 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab4120d-1345-45bb-b124-14b73465d232 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e3b28e8808774b989c36a5c3ac34dda0)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:26:35.291629 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab4120d-1345-45bb-b124-14b73465d232 admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:26:35.306959 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.auth.core [None req-f14e3826-efb0-45c6-b49c-545210e148ce None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:26:35.308174 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-f14e3826-efb0-45c6-b49c-545210e148ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:26:35.316687 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1206/2389] 10.222.0.23 () {58 vars in 957 bytes} [Thu May 19 12:26:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 12:26:35.321066 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14e3826-efb0-45c6-b49c-545210e148ce None None] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:26:35.330418 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14e3826-efb0-45c6-b49c-545210e148ce None None] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:26:35.335307 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfc964f-d711-4682-9713-de2d1d933857 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:26:35.335993 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfc964f-d711-4682-9713-de2d1d933857 admin admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:26:35.336287 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfc964f-d711-4682-9713-de2d1d933857 admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:26:35.336568 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfc964f-d711-4682-9713-de2d1d933857 admin admin] PATH_INFO: `/v3/domains` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:26:35.337310 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edfc964f-d711-4682-9713-de2d1d933857 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:26:35.337618 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edfc964f-d711-4682-9713-de2d1d933857 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:26:35.339255 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edfc964f-d711-4682-9713-de2d1d933857 admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:26:35.351754 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1207/2390] 10.222.0.23 () {58 vars in 1152 bytes} [Thu May 19 12:26:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 12:26:35.355973 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfc964f-d711-4682-9713-de2d1d933857 admin admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:26:35.366641 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfc964f-d711-4682-9713-de2d1d933857 admin admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:26:35.371232 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e88567-8db9-47d6-821c-2f6ce314339a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:26:35.376174 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e88567-8db9-47d6-821c-2f6ce314339a admin admin] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:26:35.376174 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e88567-8db9-47d6-821c-2f6ce314339a admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:26:35.376174 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e88567-8db9-47d6-821c-2f6ce314339a admin admin] PATH_INFO: `/v3/projects` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:26:35.376174 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e88567-8db9-47d6-821c-2f6ce314339a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:26:35.376174 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e88567-8db9-47d6-821c-2f6ce314339a admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:26:35.470317 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-8ab4120d-1345-45bb-b124-14b73465d232 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e3b28e8808774b989c36a5c3ac34dda0', 'request_id': 'req-8ab4120d-1345-45bb-b124-14b73465d232'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:26:35.477052 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-8ab4120d-1345-45bb-b124-14b73465d232 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e3b28e8808774b989c36a5c3ac34dda0', 'request_id': 'req-8ab4120d-1345-45bb-b124-14b73465d232'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:26:35.535759 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1208/2391] 10.222.0.23 () {60 vars in 1152 bytes} [Thu May 19 12:26:35 2022] POST /identity/v3/projects => generated 377 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 12:26:35.539844 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e88567-8db9-47d6-821c-2f6ce314339a admin admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:26:35.589289 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:26:35.604507 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e88567-8db9-47d6-821c-2f6ce314339a admin admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:26:35.610457 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce629b8b-ccb7-4bfe-99cb-43f98f81feff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:26:35.611153 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce629b8b-ccb7-4bfe-99cb-43f98f81feff admin admin] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:26:35.611488 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce629b8b-ccb7-4bfe-99cb-43f98f81feff admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:26:35.611810 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce629b8b-ccb7-4bfe-99cb-43f98f81feff admin admin] PATH_INFO: `/v3/users` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:26:35.612574 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce629b8b-ccb7-4bfe-99cb-43f98f81feff admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:26:35.614510 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce629b8b-ccb7-4bfe-99cb-43f98f81feff admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:26:35.625624 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-8ab4120d-1345-45bb-b124-14b73465d232 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e3b28e8808774b989c36a5c3ac34dda0', 'request_id': 'req-8ab4120d-1345-45bb-b124-14b73465d232'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:26:35.637840 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-8ab4120d-1345-45bb-b124-14b73465d232 admin admin] The token cache is being invalidate because project e3b28e8808774b989c36a5c3ac34dda0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80539) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 12:26:35.638166 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-8ab4120d-1345-45bb-b124-14b73465d232 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:26:35.638850 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-8ab4120d-1345-45bb-b124-14b73465d232 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:26:35.640461 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1184/2392] 10.222.0.23 () {58 vars in 1232 bytes} [Thu May 19 12:26:35 2022] DELETE /identity/v3/projects/e3b28e8808774b989c36a5c3ac34dda0 => generated 0 bytes in 433 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:26:35.645789 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b543e9f-a6de-43c2-9b9f-58ff859486b8 None None] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:26:35.646087 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b543e9f-a6de-43c2-9b9f-58ff859486b8 None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:26:35.646367 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b543e9f-a6de-43c2-9b9f-58ff859486b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:26:35.665998 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.auth.core [None req-6b543e9f-a6de-43c2-9b9f-58ff859486b8 None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:26:35.667196 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-6b543e9f-a6de-43c2-9b9f-58ff859486b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:26:35.676794 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1185/2393] 10.222.0.23 () {58 vars in 957 bytes} [Thu May 19 12:26:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 12:26:35.686411 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5803f840-f326-418f-93bf-34be9f5b3c5f None None] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:26:35.686698 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5803f840-f326-418f-93bf-34be9f5b3c5f None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:26:35.686976 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5803f840-f326-418f-93bf-34be9f5b3c5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:26:35.705143 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.auth.core [None req-5803f840-f326-418f-93bf-34be9f5b3c5f None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:26:35.707178 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-5803f840-f326-418f-93bf-34be9f5b3c5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:26:35.714463 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1209/2394] 10.222.0.23 () {60 vars in 1143 bytes} [Thu May 19 12:26:35 2022] POST /identity/v3/users => generated 289 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 12:26:35.716512 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1186/2395] 10.222.0.23 () {58 vars in 957 bytes} [Thu May 19 12:26:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 12:26:35.716645 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce629b8b-ccb7-4bfe-99cb-43f98f81feff admin admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:26:35.719092 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-ce629b8b-ccb7-4bfe-99cb-43f98f81feff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:26:35.722648 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5803f840-f326-418f-93bf-34be9f5b3c5f None None] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:26:35.731020 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce629b8b-ccb7-4bfe-99cb-43f98f81feff admin admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:26:35.731615 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5803f840-f326-418f-93bf-34be9f5b3c5f None None] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:26:35.736079 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ba37b7-5a93-4ad9-8601-22241fe57c5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:26:35.736570 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0256a45-5190-4b2f-8e05-a5d1c3e3e68c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:26:35.736745 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ba37b7-5a93-4ad9-8601-22241fe57c5a admin admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:26:35.737543 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0256a45-5190-4b2f-8e05-a5d1c3e3e68c admin admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:26:35.737662 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ba37b7-5a93-4ad9-8601-22241fe57c5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:26:35.737870 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0256a45-5190-4b2f-8e05-a5d1c3e3e68c admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:26:35.737969 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ba37b7-5a93-4ad9-8601-22241fe57c5a admin admin] PATH_INFO: `/v3/roles` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:26:35.738143 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0256a45-5190-4b2f-8e05-a5d1c3e3e68c admin admin] PATH_INFO: `/v3/domains` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:26:35.738685 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ba37b7-5a93-4ad9-8601-22241fe57c5a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:26:35.738874 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0256a45-5190-4b2f-8e05-a5d1c3e3e68c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:26:35.739179 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0256a45-5190-4b2f-8e05-a5d1c3e3e68c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:26:35.740334 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ba37b7-5a93-4ad9-8601-22241fe57c5a admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:26:35.740783 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0256a45-5190-4b2f-8e05-a5d1c3e3e68c admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:26:35.750565 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1210/2396] 10.222.0.23 () {58 vars in 1121 bytes} [Thu May 19 12:26:35 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 12:26:35.756558 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1187/2397] 10.222.0.23 () {58 vars in 1152 bytes} [Thu May 19 12:26:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 12:26:35.756704 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ba37b7-5a93-4ad9-8601-22241fe57c5a admin admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:26:35.757634 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0256a45-5190-4b2f-8e05-a5d1c3e3e68c admin admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:26:35.764912 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ba37b7-5a93-4ad9-8601-22241fe57c5a admin admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:26:35.767354 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0256a45-5190-4b2f-8e05-a5d1c3e3e68c admin admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:26:35.771111 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c58c392-b48a-4604-875b-d5e89d3f3911 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:26:35.771881 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c58c392-b48a-4604-875b-d5e89d3f3911 admin admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:26:35.772169 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c58c392-b48a-4604-875b-d5e89d3f3911 admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:26:35.772449 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c58c392-b48a-4604-875b-d5e89d3f3911 admin admin] PATH_INFO: `/v3/roles` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:26:35.773167 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c58c392-b48a-4604-875b-d5e89d3f3911 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:26:35.773704 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9a9643-e4d5-425b-9a3b-f2eff1c16395 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:26:35.774362 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9a9643-e4d5-425b-9a3b-f2eff1c16395 admin admin] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:26:35.774674 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9a9643-e4d5-425b-9a3b-f2eff1c16395 admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:26:35.774786 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c58c392-b48a-4604-875b-d5e89d3f3911 admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:26:35.774946 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9a9643-e4d5-425b-9a3b-f2eff1c16395 admin admin] PATH_INFO: `/v3/projects` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:26:35.775646 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9a9643-e4d5-425b-9a3b-f2eff1c16395 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:26:35.777764 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9a9643-e4d5-425b-9a3b-f2eff1c16395 admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:26:35.786751 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1211/2398] 10.222.0.23 () {58 vars in 1121 bytes} [Thu May 19 12:26:35 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 19 12:26:35.792014 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c58c392-b48a-4604-875b-d5e89d3f3911 admin admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:26:35.802560 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c58c392-b48a-4604-875b-d5e89d3f3911 admin admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:26:35.807414 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17c2ba6-16ce-484a-b4bf-ffed3a52dcfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:26:35.812057 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17c2ba6-16ce-484a-b4bf-ffed3a52dcfa admin admin] REQUEST_METHOD: `PUT` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:26:35.812057 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17c2ba6-16ce-484a-b4bf-ffed3a52dcfa admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:26:35.812057 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17c2ba6-16ce-484a-b4bf-ffed3a52dcfa admin admin] PATH_INFO: `/v3/projects/4b46044a7f1d471582800c9d8a79e5c5/users/2d158b01a0a74de4bc0131b5c625efdf/roles/259eb400cdb4439db72d14456ae9cd08` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:26:35.825289 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17c2ba6-16ce-484a-b4bf-ffed3a52dcfa admin admin] RBAC: Authorizing `identity:create_grant(project_id=4b46044a7f1d471582800c9d8a79e5c5, user_id=2d158b01a0a74de4bc0131b5c625efdf, role_id=259eb400cdb4439db72d14456ae9cd08)` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:26:35.828045 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17c2ba6-16ce-484a-b4bf-ffed3a52dcfa admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:26:35.981047 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1188/2399] 10.222.0.23 () {60 vars in 1152 bytes} [Thu May 19 12:26:35 2022] POST /identity/v3/projects => generated 375 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 12:26:35.985305 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9a9643-e4d5-425b-9a3b-f2eff1c16395 admin admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:26:36.025211 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1212/2400] 10.222.0.23 () {60 vars in 1482 bytes} [Thu May 19 12:26:35 2022] PUT /identity/v3/projects/4b46044a7f1d471582800c9d8a79e5c5/users/2d158b01a0a74de4bc0131b5c625efdf/roles/259eb400cdb4439db72d14456ae9cd08 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 12:26:36.030102 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55a7db0-21cd-4b18-96ef-9aaa7c826ac2 None None] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:26:36.030420 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55a7db0-21cd-4b18-96ef-9aaa7c826ac2 None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:26:36.030829 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55a7db0-21cd-4b18-96ef-9aaa7c826ac2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:26:36.047212 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9a9643-e4d5-425b-9a3b-f2eff1c16395 admin admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:26:36.048797 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.auth.core [None req-c55a7db0-21cd-4b18-96ef-9aaa7c826ac2 None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:26:36.050936 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-c55a7db0-21cd-4b18-96ef-9aaa7c826ac2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:26:36.104349 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f040cb49-c6a6-4ac3-b926-130f36bcdb21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:26:36.105029 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f040cb49-c6a6-4ac3-b926-130f36bcdb21 admin admin] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:26:36.105336 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f040cb49-c6a6-4ac3-b926-130f36bcdb21 admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:26:36.105613 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f040cb49-c6a6-4ac3-b926-130f36bcdb21 admin admin] PATH_INFO: `/v3/users` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:26:36.106341 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f040cb49-c6a6-4ac3-b926-130f36bcdb21 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:26:36.108439 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f040cb49-c6a6-4ac3-b926-130f36bcdb21 admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:26:36.110036 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1213/2401] 10.222.0.23 () {58 vars in 957 bytes} [Thu May 19 12:26:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 12:26:36.120418 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55a7db0-21cd-4b18-96ef-9aaa7c826ac2 None None] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:26:36.121844 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-c55a7db0-21cd-4b18-96ef-9aaa7c826ac2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:26:36.133288 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:26:36.179708 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55a7db0-21cd-4b18-96ef-9aaa7c826ac2 None None] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:26:36.183437 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1aa870-502d-4f87-bcd0-0d6575ef66bc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a00b3fa7497456895e5e0bf105c4fc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd70d25bf69314820b8aaecb65576978a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:26:36.188903 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1aa870-502d-4f87-bcd0-0d6575ef66bc service neutron] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:26:36.188903 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1aa870-502d-4f87-bcd0-0d6575ef66bc service neutron] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:26:36.188903 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1aa870-502d-4f87-bcd0-0d6575ef66bc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:26:36.194089 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1aa870-502d-4f87-bcd0-0d6575ef66bc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:26:36.195589 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1aa870-502d-4f87-bcd0-0d6575ef66bc service neutron] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:26:36.205145 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1214/2402] 10.222.0.23 () {60 vars in 1365 bytes} [Thu May 19 12:26:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 12:26:36.226756 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1189/2403] 10.222.0.23 () {60 vars in 1143 bytes} [Thu May 19 12:26:35 2022] POST /identity/v3/users => generated 288 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 12:26:36.229574 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1aa870-502d-4f87-bcd0-0d6575ef66bc service neutron] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:26:36.239391 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1aa870-502d-4f87-bcd0-0d6575ef66bc service neutron] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:26:36.244194 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169e1962-a634-4950-8307-e69f9cb92e40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:26:36.244852 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169e1962-a634-4950-8307-e69f9cb92e40 admin admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:26:36.245139 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169e1962-a634-4950-8307-e69f9cb92e40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:26:36.245419 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169e1962-a634-4950-8307-e69f9cb92e40 admin admin] PATH_INFO: `/v3/roles` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:26:36.246141 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169e1962-a634-4950-8307-e69f9cb92e40 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:26:36.247823 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169e1962-a634-4950-8307-e69f9cb92e40 admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:26:36.258330 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1215/2404] 10.222.0.23 () {58 vars in 1121 bytes} [Thu May 19 12:26:36 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 12:26:36.271804 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f040cb49-c6a6-4ac3-b926-130f36bcdb21 admin admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:26:36.281919 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f040cb49-c6a6-4ac3-b926-130f36bcdb21 admin admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:26:36.287091 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee4d4da-2989-4f2a-80d5-b153cd46ceea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:26:36.288117 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee4d4da-2989-4f2a-80d5-b153cd46ceea admin admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:26:36.288255 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee4d4da-2989-4f2a-80d5-b153cd46ceea admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:26:36.288493 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee4d4da-2989-4f2a-80d5-b153cd46ceea admin admin] PATH_INFO: `/v3/roles` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:26:36.289224 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee4d4da-2989-4f2a-80d5-b153cd46ceea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:26:36.290819 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee4d4da-2989-4f2a-80d5-b153cd46ceea admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:26:36.301094 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1190/2405] 10.222.0.23 () {58 vars in 1121 bytes} [Thu May 19 12:26:36 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 12:26:36.305352 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169e1962-a634-4950-8307-e69f9cb92e40 admin admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:26:36.315917 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169e1962-a634-4950-8307-e69f9cb92e40 admin admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:26:36.323323 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49bc679a-28d3-4c3d-a532-2b4231216f67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:26:36.323323 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bc679a-28d3-4c3d-a532-2b4231216f67 admin admin] REQUEST_METHOD: `PUT` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:26:36.323323 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bc679a-28d3-4c3d-a532-2b4231216f67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:26:36.323323 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bc679a-28d3-4c3d-a532-2b4231216f67 admin admin] PATH_INFO: `/v3/projects/22dfcf3b0aaf411cba54d8badcd319f4/users/a60bf7d09fcf4e5c91f07b43f69aefa8/roles/259eb400cdb4439db72d14456ae9cd08` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:26:36.338633 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49bc679a-28d3-4c3d-a532-2b4231216f67 admin admin] RBAC: Authorizing `identity:create_grant(project_id=22dfcf3b0aaf411cba54d8badcd319f4, user_id=a60bf7d09fcf4e5c91f07b43f69aefa8, role_id=259eb400cdb4439db72d14456ae9cd08)` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:26:36.341351 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49bc679a-28d3-4c3d-a532-2b4231216f67 admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:26:36.489310 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:26:36.495074 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1216/2406] 10.222.0.23 () {60 vars in 1482 bytes} [Thu May 19 12:26:36 2022] PUT /identity/v3/projects/22dfcf3b0aaf411cba54d8badcd319f4/users/a60bf7d09fcf4e5c91f07b43f69aefa8/roles/259eb400cdb4439db72d14456ae9cd08 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:26:36.506777 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b2fd0c-a8c2-4e43-a43c-da69d942b5f2 None None] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:26:36.507088 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b2fd0c-a8c2-4e43-a43c-da69d942b5f2 None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:26:36.507375 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b2fd0c-a8c2-4e43-a43c-da69d942b5f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:26:36.536985 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.auth.core [None req-d6b2fd0c-a8c2-4e43-a43c-da69d942b5f2 None None] MFA Rules not processed for user `a60bf7d09fcf4e5c91f07b43f69aefa8`. Rule list: `[]` (Enabled: `True`). {{(pid=80539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:26:36.538221 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-d6b2fd0c-a8c2-4e43-a43c-da69d942b5f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:26:36.606151 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1191/2407] 10.222.0.23 () {58 vars in 957 bytes} [Thu May 19 12:26:36 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 12:26:36.617629 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49bc679a-28d3-4c3d-a532-2b4231216f67 admin admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:26:36.620712 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-49bc679a-28d3-4c3d-a532-2b4231216f67 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:26:36.683473 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49bc679a-28d3-4c3d-a532-2b4231216f67 admin admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:26:36.687165 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab671fd-65d7-4560-a8a5-be95fbe65425 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8800254c0e784e42a791fb9abc274d79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd70d25bf69314820b8aaecb65576978a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:26:36.687834 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab671fd-65d7-4560-a8a5-be95fbe65425 service cinder] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:26:36.688122 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab671fd-65d7-4560-a8a5-be95fbe65425 service cinder] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:26:36.688401 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab671fd-65d7-4560-a8a5-be95fbe65425 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:26:36.697374 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab671fd-65d7-4560-a8a5-be95fbe65425 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:26:36.699036 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab671fd-65d7-4560-a8a5-be95fbe65425 service cinder] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:26:36.710101 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1217/2408] 10.222.0.23 () {60 vars in 1365 bytes} [Thu May 19 12:26:36 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:26:42.379397 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b2fd0c-a8c2-4e43-a43c-da69d942b5f2 None None] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:26:42.456415 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:26:42.488095 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b2fd0c-a8c2-4e43-a43c-da69d942b5f2 None None] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:26:42.494074 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2259834e-7c81-47f8-9923-254964243069 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:26:42.495361 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2259834e-7c81-47f8-9923-254964243069 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:26:42.496067 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2259834e-7c81-47f8-9923-254964243069 admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:26:42.496591 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2259834e-7c81-47f8-9923-254964243069 admin admin] PATH_INFO: `/v3/users/a60bf7d09fcf4e5c91f07b43f69aefa8` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:26:42.498579 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2259834e-7c81-47f8-9923-254964243069 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a60bf7d09fcf4e5c91f07b43f69aefa8)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:26:42.500899 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2259834e-7c81-47f8-9923-254964243069 admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:26:42.609898 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d000fd-dfcb-43e9-b863-0c28198d6bba None None] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:26:42.610431 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d000fd-dfcb-43e9-b863-0c28198d6bba None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:26:42.610431 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d000fd-dfcb-43e9-b863-0c28198d6bba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:26:42.638629 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.auth.core [None req-63d000fd-dfcb-43e9-b863-0c28198d6bba None None] MFA Rules not processed for user `2d158b01a0a74de4bc0131b5c625efdf`. Rule list: `[]` (Enabled: `True`). {{(pid=80540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:26:42.639878 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-63d000fd-dfcb-43e9-b863-0c28198d6bba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:26:42.717813 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1218/2409] 10.222.0.23 () {58 vars in 957 bytes} [Thu May 19 12:26:42 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 12:26:42.722647 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d000fd-dfcb-43e9-b863-0c28198d6bba None None] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:26:42.733218 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d000fd-dfcb-43e9-b863-0c28198d6bba None None] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:26:42.739510 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74be5f8f-5b02-4879-97a9-184c75272c5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:26:42.740259 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74be5f8f-5b02-4879-97a9-184c75272c5d admin admin] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:26:42.740549 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74be5f8f-5b02-4879-97a9-184c75272c5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:26:42.740852 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74be5f8f-5b02-4879-97a9-184c75272c5d admin admin] PATH_INFO: `/v3/projects` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:26:42.741545 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74be5f8f-5b02-4879-97a9-184c75272c5d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:26:42.743488 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74be5f8f-5b02-4879-97a9-184c75272c5d admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:26:42.764911 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-2259834e-7c81-47f8-9923-254964243069 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a60bf7d09fcf4e5c91f07b43f69aefa8'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:26:42.777058 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-2259834e-7c81-47f8-9923-254964243069 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a60bf7d09fcf4e5c91f07b43f69aefa8'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:26:42.777764 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-2259834e-7c81-47f8-9923-254964243069 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a60bf7d09fcf4e5c91f07b43f69aefa8'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:26:42.924961 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1219/2410] 10.222.0.23 () {60 vars in 1152 bytes} [Thu May 19 12:26:42 2022] POST /identity/v3/projects => generated 377 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 12:26:42.932124 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74be5f8f-5b02-4879-97a9-184c75272c5d admin admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:26:42.979003 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-2259834e-7c81-47f8-9923-254964243069 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a60bf7d09fcf4e5c91f07b43f69aefa8'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:26:42.987759 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-2259834e-7c81-47f8-9923-254964243069 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a60bf7d09fcf4e5c91f07b43f69aefa8'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:26:42.993970 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1192/2411] 10.222.0.23 () {58 vars in 1223 bytes} [Thu May 19 12:26:42 2022] DELETE /identity/v3/users/a60bf7d09fcf4e5c91f07b43f69aefa8 => generated 0 bytes in 615 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:26:42.998275 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74be5f8f-5b02-4879-97a9-184c75272c5d admin admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:26:42.998504 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459cb4fa-c36e-4a96-94f4-a1a528f984ae None None] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:26:42.998788 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459cb4fa-c36e-4a96-94f4-a1a528f984ae None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:26:42.999067 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459cb4fa-c36e-4a96-94f4-a1a528f984ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:26:43.016125 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.auth.core [None req-459cb4fa-c36e-4a96-94f4-a1a528f984ae None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:26:43.016652 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:26:43.017333 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-459cb4fa-c36e-4a96-94f4-a1a528f984ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:26:43.067914 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7141ad0c-80be-49fd-9015-0e0bf087acec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:26:43.068715 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7141ad0c-80be-49fd-9015-0e0bf087acec admin admin] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:26:43.068950 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7141ad0c-80be-49fd-9015-0e0bf087acec admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:26:43.069332 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7141ad0c-80be-49fd-9015-0e0bf087acec admin admin] PATH_INFO: `/v3/users` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:26:43.069923 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7141ad0c-80be-49fd-9015-0e0bf087acec admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:26:43.071966 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7141ad0c-80be-49fd-9015-0e0bf087acec admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:26:43.081869 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1193/2412] 10.222.0.23 () {58 vars in 957 bytes} [Thu May 19 12:26:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:26:43.092217 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459cb4fa-c36e-4a96-94f4-a1a528f984ae None None] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:26:43.093707 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-459cb4fa-c36e-4a96-94f4-a1a528f984ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:26:43.161605 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459cb4fa-c36e-4a96-94f4-a1a528f984ae None None] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:26:43.165466 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c1b733-88ab-41f3-b221-0d1b3d81c7b1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a00b3fa7497456895e5e0bf105c4fc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd70d25bf69314820b8aaecb65576978a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:26:43.166113 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c1b733-88ab-41f3-b221-0d1b3d81c7b1 service neutron] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:26:43.166399 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c1b733-88ab-41f3-b221-0d1b3d81c7b1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:26:43.166678 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c1b733-88ab-41f3-b221-0d1b3d81c7b1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:26:43.176975 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c1b733-88ab-41f3-b221-0d1b3d81c7b1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:26:43.178394 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c1b733-88ab-41f3-b221-0d1b3d81c7b1 service neutron] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:26:43.184174 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:26:43.189416 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1194/2413] 10.222.0.23 () {60 vars in 1365 bytes} [Thu May 19 12:26:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:26:43.203716 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1220/2414] 10.222.0.23 () {60 vars in 1143 bytes} [Thu May 19 12:26:42 2022] POST /identity/v3/users => generated 295 bytes in 270 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 12:26:43.205092 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c1b733-88ab-41f3-b221-0d1b3d81c7b1 service neutron] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:26:43.207114 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-82c1b733-88ab-41f3-b221-0d1b3d81c7b1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:26:43.220472 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c1b733-88ab-41f3-b221-0d1b3d81c7b1 service neutron] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:26:43.226182 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a712718e-8a07-4794-b71a-cdd64edc99d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:26:43.226895 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a712718e-8a07-4794-b71a-cdd64edc99d3 admin admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:26:43.227181 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a712718e-8a07-4794-b71a-cdd64edc99d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:26:43.227460 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a712718e-8a07-4794-b71a-cdd64edc99d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:26:43.228241 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a712718e-8a07-4794-b71a-cdd64edc99d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:26:43.230279 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a712718e-8a07-4794-b71a-cdd64edc99d3 admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:26:43.248718 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1195/2415] 10.222.0.23 () {58 vars in 1121 bytes} [Thu May 19 12:26:43 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 19 12:26:43.250381 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7141ad0c-80be-49fd-9015-0e0bf087acec admin admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:26:43.263662 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7141ad0c-80be-49fd-9015-0e0bf087acec admin admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:26:43.269571 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503c50bd-d9c9-4269-b970-d072a243f389 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:26:43.270508 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503c50bd-d9c9-4269-b970-d072a243f389 admin admin] REQUEST_METHOD: `PUT` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:26:43.270797 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503c50bd-d9c9-4269-b970-d072a243f389 admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:26:43.271091 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503c50bd-d9c9-4269-b970-d072a243f389 admin admin] PATH_INFO: `/v3/projects/fe5c73250c7c4b96920c3d5ec5678ff5/users/2d9f8b22c6e44f1784ac4ca285cfbcf5/roles/9db5671c65d3414ea2682afa2ccf1403` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:26:43.290071 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503c50bd-d9c9-4269-b970-d072a243f389 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fe5c73250c7c4b96920c3d5ec5678ff5, user_id=2d9f8b22c6e44f1784ac4ca285cfbcf5, role_id=9db5671c65d3414ea2682afa2ccf1403)` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:26:43.292884 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503c50bd-d9c9-4269-b970-d072a243f389 admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:26:43.385984 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1221/2416] 10.222.0.23 () {60 vars in 1482 bytes} [Thu May 19 12:26:43 2022] PUT /identity/v3/projects/fe5c73250c7c4b96920c3d5ec5678ff5/users/2d9f8b22c6e44f1784ac4ca285cfbcf5/roles/9db5671c65d3414ea2682afa2ccf1403 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:26:43.389969 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a712718e-8a07-4794-b71a-cdd64edc99d3 admin admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:26:43.401816 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a712718e-8a07-4794-b71a-cdd64edc99d3 admin admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:26:43.476346 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e47576b-6b72-482d-9ccb-23a3c2afec23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:26:43.477198 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e47576b-6b72-482d-9ccb-23a3c2afec23 admin admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:26:43.477372 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e47576b-6b72-482d-9ccb-23a3c2afec23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:26:43.477651 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e47576b-6b72-482d-9ccb-23a3c2afec23 admin admin] PATH_INFO: `/v3/roles` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:26:43.478463 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e47576b-6b72-482d-9ccb-23a3c2afec23 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:26:43.480523 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e47576b-6b72-482d-9ccb-23a3c2afec23 admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:26:43.492704 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1196/2417] 10.222.0.23 () {58 vars in 1121 bytes} [Thu May 19 12:26:43 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 12:26:43.496629 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503c50bd-d9c9-4269-b970-d072a243f389 admin admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:26:43.508174 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503c50bd-d9c9-4269-b970-d072a243f389 admin admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:26:43.513297 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c4a377-845b-486b-b298-a43e23ee11d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:26:43.514595 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c4a377-845b-486b-b298-a43e23ee11d2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:26:43.514719 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c4a377-845b-486b-b298-a43e23ee11d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:26:43.514941 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c4a377-845b-486b-b298-a43e23ee11d2 admin admin] PATH_INFO: `/v3/projects/fe5c73250c7c4b96920c3d5ec5678ff5/users/2d9f8b22c6e44f1784ac4ca285cfbcf5/roles/9db5671c65d3414ea2682afa2ccf1403` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:26:43.517029 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c4a377-845b-486b-b298-a43e23ee11d2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fe5c73250c7c4b96920c3d5ec5678ff5, user_id=2d9f8b22c6e44f1784ac4ca285cfbcf5, role_id=9db5671c65d3414ea2682afa2ccf1403)` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:26:43.520451 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c4a377-845b-486b-b298-a43e23ee11d2 admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:26:43.533624 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1222/2418] 10.222.0.23 () {60 vars in 1482 bytes} [Thu May 19 12:26:43 2022] PUT /identity/v3/projects/fe5c73250c7c4b96920c3d5ec5678ff5/users/2d9f8b22c6e44f1784ac4ca285cfbcf5/roles/9db5671c65d3414ea2682afa2ccf1403 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:26:43.687026 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e47576b-6b72-482d-9ccb-23a3c2afec23 admin admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:26:43.692780 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-4e47576b-6b72-482d-9ccb-23a3c2afec23 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:26:43.712119 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e47576b-6b72-482d-9ccb-23a3c2afec23 admin admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:26:43.816851 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e648ba4-9fd7-4dfc-8351-459b583951e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:26:43.817888 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e648ba4-9fd7-4dfc-8351-459b583951e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:26:43.818238 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e648ba4-9fd7-4dfc-8351-459b583951e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:26:43.818586 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e648ba4-9fd7-4dfc-8351-459b583951e1 admin admin] PATH_INFO: `/v3/projects/22dfcf3b0aaf411cba54d8badcd319f4` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:26:43.820320 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e648ba4-9fd7-4dfc-8351-459b583951e1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=22dfcf3b0aaf411cba54d8badcd319f4)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:26:43.822366 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e648ba4-9fd7-4dfc-8351-459b583951e1 admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:26:44.113902 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-1e648ba4-9fd7-4dfc-8351-459b583951e1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '22dfcf3b0aaf411cba54d8badcd319f4', 'request_id': 'req-1e648ba4-9fd7-4dfc-8351-459b583951e1'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:26:44.120669 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-1e648ba4-9fd7-4dfc-8351-459b583951e1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '22dfcf3b0aaf411cba54d8badcd319f4', 'request_id': 'req-1e648ba4-9fd7-4dfc-8351-459b583951e1'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:26:44.276611 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-1e648ba4-9fd7-4dfc-8351-459b583951e1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '22dfcf3b0aaf411cba54d8badcd319f4', 'request_id': 'req-1e648ba4-9fd7-4dfc-8351-459b583951e1'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:26:44.294673 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-1e648ba4-9fd7-4dfc-8351-459b583951e1 admin admin] The token cache is being invalidate because project 22dfcf3b0aaf411cba54d8badcd319f4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80539) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 12:26:44.294673 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-1e648ba4-9fd7-4dfc-8351-459b583951e1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:26:44.295335 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-1e648ba4-9fd7-4dfc-8351-459b583951e1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:26:44.297587 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1197/2419] 10.222.0.23 () {58 vars in 1232 bytes} [Thu May 19 12:26:43 2022] DELETE /identity/v3/projects/22dfcf3b0aaf411cba54d8badcd319f4 => generated 0 bytes in 611 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:26:44.302475 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbfdabe-faca-4d7c-a310-af0ae88a26ab None None] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:26:44.302983 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbfdabe-faca-4d7c-a310-af0ae88a26ab None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:26:44.303171 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbfdabe-faca-4d7c-a310-af0ae88a26ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:26:44.322444 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.auth.core [None req-1bbfdabe-faca-4d7c-a310-af0ae88a26ab None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:26:44.323662 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-1bbfdabe-faca-4d7c-a310-af0ae88a26ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:26:44.420318 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1223/2420] 10.222.0.23 () {58 vars in 957 bytes} [Thu May 19 12:26:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 12:26:44.430044 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20186e63-27e4-48eb-997e-de2d2d0e946a None None] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:26:44.430617 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20186e63-27e4-48eb-997e-de2d2d0e946a None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:26:44.430722 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20186e63-27e4-48eb-997e-de2d2d0e946a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:26:44.453711 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.auth.core [None req-20186e63-27e4-48eb-997e-de2d2d0e946a None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:26:44.455130 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-20186e63-27e4-48eb-997e-de2d2d0e946a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:26:44.464534 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1198/2421] 10.222.0.23 () {58 vars in 957 bytes} [Thu May 19 12:26:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:26:44.480998 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbfdabe-faca-4d7c-a310-af0ae88a26ab None None] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:26:44.492186 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbfdabe-faca-4d7c-a310-af0ae88a26ab None None] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:26:44.496935 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ac99e4-cd20-4f73-ad97-6a31cad60279 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:26:44.497598 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ac99e4-cd20-4f73-ad97-6a31cad60279 admin admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:26:44.497890 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ac99e4-cd20-4f73-ad97-6a31cad60279 admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:26:44.498175 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ac99e4-cd20-4f73-ad97-6a31cad60279 admin admin] PATH_INFO: `/v3/domains` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:26:44.498915 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ac99e4-cd20-4f73-ad97-6a31cad60279 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:26:44.499239 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ac99e4-cd20-4f73-ad97-6a31cad60279 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:26:44.500903 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ac99e4-cd20-4f73-ad97-6a31cad60279 admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:26:44.513543 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1224/2422] 10.222.0.23 () {58 vars in 1152 bytes} [Thu May 19 12:26:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:26:44.518701 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20186e63-27e4-48eb-997e-de2d2d0e946a None None] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:26:44.529554 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20186e63-27e4-48eb-997e-de2d2d0e946a None None] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:26:44.534954 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ea6963-a56e-41c3-a261-d24567480960 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:26:44.535627 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ea6963-a56e-41c3-a261-d24567480960 admin admin] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:26:44.535951 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ea6963-a56e-41c3-a261-d24567480960 admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:26:44.536238 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ea6963-a56e-41c3-a261-d24567480960 admin admin] PATH_INFO: `/v3/projects` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:26:44.536944 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ea6963-a56e-41c3-a261-d24567480960 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:26:44.538916 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ea6963-a56e-41c3-a261-d24567480960 admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:26:44.544257 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:26:44.660845 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1199/2423] 10.222.0.23 () {60 vars in 1152 bytes} [Thu May 19 12:26:44 2022] POST /identity/v3/projects => generated 369 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 12:26:44.668760 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ac99e4-cd20-4f73-ad97-6a31cad60279 admin admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:26:44.677949 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ac99e4-cd20-4f73-ad97-6a31cad60279 admin admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:26:44.755553 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:26:44.767847 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f4498c-45ed-422c-bc2f-f63dcaeaa91f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:26:44.768986 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f4498c-45ed-422c-bc2f-f63dcaeaa91f admin admin] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:26:44.769283 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f4498c-45ed-422c-bc2f-f63dcaeaa91f admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:26:44.769567 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f4498c-45ed-422c-bc2f-f63dcaeaa91f admin admin] PATH_INFO: `/v3/users` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:26:44.773168 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f4498c-45ed-422c-bc2f-f63dcaeaa91f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:26:44.773168 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f4498c-45ed-422c-bc2f-f63dcaeaa91f admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:26:44.909003 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1225/2424] 10.222.0.23 () {60 vars in 1143 bytes} [Thu May 19 12:26:44 2022] POST /identity/v3/users => generated 285 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 12:26:44.910438 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ea6963-a56e-41c3-a261-d24567480960 admin admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:26:44.921369 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ea6963-a56e-41c3-a261-d24567480960 admin admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:26:44.926207 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55b113b-7313-4d4b-87b9-164ca94863a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:26:44.927861 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55b113b-7313-4d4b-87b9-164ca94863a8 admin admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:26:44.927861 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55b113b-7313-4d4b-87b9-164ca94863a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:26:44.927861 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55b113b-7313-4d4b-87b9-164ca94863a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:26:44.931862 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55b113b-7313-4d4b-87b9-164ca94863a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:26:44.931862 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55b113b-7313-4d4b-87b9-164ca94863a8 admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:26:44.949444 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1200/2425] 10.222.0.23 () {58 vars in 1121 bytes} [Thu May 19 12:26:44 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 12:26:44.953045 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f4498c-45ed-422c-bc2f-f63dcaeaa91f admin admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:26:44.967159 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f4498c-45ed-422c-bc2f-f63dcaeaa91f admin admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:26:44.973613 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e17f78-7b29-4961-98f7-4519dca51ddf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:26:44.974379 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e17f78-7b29-4961-98f7-4519dca51ddf admin admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:26:44.974665 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e17f78-7b29-4961-98f7-4519dca51ddf admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:26:44.974958 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e17f78-7b29-4961-98f7-4519dca51ddf admin admin] PATH_INFO: `/v3/roles` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:26:44.977073 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e17f78-7b29-4961-98f7-4519dca51ddf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:26:44.978890 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e17f78-7b29-4961-98f7-4519dca51ddf admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:26:45.001537 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1226/2426] 10.222.0.23 () {58 vars in 1121 bytes} [Thu May 19 12:26:44 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 12:26:45.009003 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55b113b-7313-4d4b-87b9-164ca94863a8 admin admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:26:45.027957 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55b113b-7313-4d4b-87b9-164ca94863a8 admin admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:26:45.036308 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb47a76f-70af-4702-be49-ea0b633c9133 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:26:45.037373 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb47a76f-70af-4702-be49-ea0b633c9133 admin admin] REQUEST_METHOD: `PUT` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:26:45.037661 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb47a76f-70af-4702-be49-ea0b633c9133 admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:26:45.038061 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb47a76f-70af-4702-be49-ea0b633c9133 admin admin] PATH_INFO: `/v3/projects/0a22db288abd45aba333be3640ead9ee/users/577e68f0e5fa48f49696e6a511698c50/roles/259eb400cdb4439db72d14456ae9cd08` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:26:45.058485 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb47a76f-70af-4702-be49-ea0b633c9133 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0a22db288abd45aba333be3640ead9ee, user_id=577e68f0e5fa48f49696e6a511698c50, role_id=259eb400cdb4439db72d14456ae9cd08)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:26:45.061268 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb47a76f-70af-4702-be49-ea0b633c9133 admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:26:45.222547 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1201/2427] 10.222.0.23 () {60 vars in 1482 bytes} [Thu May 19 12:26:45 2022] PUT /identity/v3/projects/0a22db288abd45aba333be3640ead9ee/users/577e68f0e5fa48f49696e6a511698c50/roles/259eb400cdb4439db72d14456ae9cd08 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:26:45.231578 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2175ce79-9b4d-49db-a657-c2719804a49d None None] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:26:45.231911 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2175ce79-9b4d-49db-a657-c2719804a49d None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:26:45.232200 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2175ce79-9b4d-49db-a657-c2719804a49d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:26:45.262326 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.auth.core [None req-2175ce79-9b4d-49db-a657-c2719804a49d None None] MFA Rules not processed for user `577e68f0e5fa48f49696e6a511698c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:26:45.263528 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-2175ce79-9b4d-49db-a657-c2719804a49d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:26:45.341438 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1227/2428] 10.222.0.23 () {58 vars in 957 bytes} [Thu May 19 12:26:45 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:26:45.354686 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb47a76f-70af-4702-be49-ea0b633c9133 admin admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:26:45.356474 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-eb47a76f-70af-4702-be49-ea0b633c9133 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:26:45.428814 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb47a76f-70af-4702-be49-ea0b633c9133 admin admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:26:45.432761 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f44f34-712a-4e20-a5b7-ebcc3aa6b08b service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '21697802f2be42f9baa8e07490d5ebdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd70d25bf69314820b8aaecb65576978a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:26:45.434165 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f44f34-712a-4e20-a5b7-ebcc3aa6b08b service glance] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:26:45.434705 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f44f34-712a-4e20-a5b7-ebcc3aa6b08b service glance] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:26:45.435141 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f44f34-712a-4e20-a5b7-ebcc3aa6b08b service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:26:45.445525 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f44f34-712a-4e20-a5b7-ebcc3aa6b08b service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:26:45.447013 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f44f34-712a-4e20-a5b7-ebcc3aa6b08b service glance] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:26:45.458556 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1202/2429] 10.222.0.23 () {60 vars in 1365 bytes} [Thu May 19 12:26:45 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:26:48.306347 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2175ce79-9b4d-49db-a657-c2719804a49d None None] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:26:48.308057 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-2175ce79-9b4d-49db-a657-c2719804a49d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:26:48.498385 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2175ce79-9b4d-49db-a657-c2719804a49d None None] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:26:48.515294 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af82bbcd-f7fa-456a-9fd2-b36238eb2c39 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50f937eed4f3414fa6f19cab2a84eb5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd70d25bf69314820b8aaecb65576978a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:26:48.517095 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af82bbcd-f7fa-456a-9fd2-b36238eb2c39 service placement] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:26:48.517411 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af82bbcd-f7fa-456a-9fd2-b36238eb2c39 service placement] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:26:48.517698 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af82bbcd-f7fa-456a-9fd2-b36238eb2c39 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:26:48.520174 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-af82bbcd-f7fa-456a-9fd2-b36238eb2c39 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:26:48.540265 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af82bbcd-f7fa-456a-9fd2-b36238eb2c39 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:26:48.546861 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af82bbcd-f7fa-456a-9fd2-b36238eb2c39 service placement] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:26:48.560536 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1228/2430] 10.222.0.23 () {60 vars in 1365 bytes} [Thu May 19 12:26:48 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 255 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:26:50.535910 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70afa7fd-fce4-43d1-b9c2-7e5fdf3c96ec None None] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:26:50.537125 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70afa7fd-fce4-43d1-b9c2-7e5fdf3c96ec None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:26:50.537371 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70afa7fd-fce4-43d1-b9c2-7e5fdf3c96ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:26:50.570762 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.auth.core [None req-70afa7fd-fce4-43d1-b9c2-7e5fdf3c96ec None None] MFA Rules not processed for user `2d9f8b22c6e44f1784ac4ca285cfbcf5`. Rule list: `[]` (Enabled: `True`). {{(pid=80539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:26:50.572149 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-70afa7fd-fce4-43d1-b9c2-7e5fdf3c96ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:26:50.648741 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1203/2431] 10.222.0.23 () {58 vars in 957 bytes} [Thu May 19 12:26:50 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:26:50.664497 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af82bbcd-f7fa-456a-9fd2-b36238eb2c39 service placement] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:26:50.665941 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-af82bbcd-f7fa-456a-9fd2-b36238eb2c39 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:26:50.728818 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:26:50.751535 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af82bbcd-f7fa-456a-9fd2-b36238eb2c39 service placement] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:26:50.758878 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4368d6-a260-443d-b205-55322537be7f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34a98d75d84b4c67981377e0f26eca62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd70d25bf69314820b8aaecb65576978a', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:26:50.760267 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4368d6-a260-443d-b205-55322537be7f service nova] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:26:50.760697 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4368d6-a260-443d-b205-55322537be7f service nova] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:26:50.761155 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4368d6-a260-443d-b205-55322537be7f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:26:50.763625 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-0d4368d6-a260-443d-b205-55322537be7f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:26:50.834243 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4368d6-a260-443d-b205-55322537be7f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:26:50.836063 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4368d6-a260-443d-b205-55322537be7f service nova] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:26:50.857269 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1229/2432] 10.222.0.23 () {60 vars in 1365 bytes} [Thu May 19 12:26:50 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:27:00.645627 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=80,keystone:INSERT=2 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:27:01.074033 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=32 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:27:31.928517 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95ea417-d431-4430-aaa3-f2d084931bd6 None None] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:27:31.928517 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95ea417-d431-4430-aaa3-f2d084931bd6 None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:27:31.928517 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95ea417-d431-4430-aaa3-f2d084931bd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:27:31.952627 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.auth.core [None req-b95ea417-d431-4430-aaa3-f2d084931bd6 None None] MFA Rules not processed for user `2d158b01a0a74de4bc0131b5c625efdf`. Rule list: `[]` (Enabled: `True`). {{(pid=80539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:27:31.953294 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-b95ea417-d431-4430-aaa3-f2d084931bd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:27:31.960718 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1204/2433] 10.222.0.23 () {58 vars in 957 bytes} [Thu May 19 12:27:31 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:27:41.148671 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c89272-387e-4b02-90ae-fb918ab47d43 None None] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:27:41.153354 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c89272-387e-4b02-90ae-fb918ab47d43 None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:27:41.153354 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c89272-387e-4b02-90ae-fb918ab47d43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:27:41.167302 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.auth.core [None req-79c89272-387e-4b02-90ae-fb918ab47d43 None None] MFA Rules not processed for user `dd39ddcf4ab045b98a737ee529a2dcce`. Rule list: `[]` (Enabled: `True`). {{(pid=80540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:27:41.168535 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-79c89272-387e-4b02-90ae-fb918ab47d43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:27:41.233991 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1230/2434] 10.222.0.23 () {58 vars in 957 bytes} [Thu May 19 12:27:41 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:27:41.942430 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=3 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:27:44.235667 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95ea417-d431-4430-aaa3-f2d084931bd6 None None] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:27:44.238147 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-b95ea417-d431-4430-aaa3-f2d084931bd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:27:44.358335 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95ea417-d431-4430-aaa3-f2d084931bd6 None None] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:27:44.365278 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6d60bb-1f06-42fa-b941-37ba37e81db3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a00b3fa7497456895e5e0bf105c4fc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd70d25bf69314820b8aaecb65576978a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:27:44.366678 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6d60bb-1f06-42fa-b941-37ba37e81db3 service neutron] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:27:44.367311 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6d60bb-1f06-42fa-b941-37ba37e81db3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:27:44.367927 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6d60bb-1f06-42fa-b941-37ba37e81db3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:27:44.372062 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-1f6d60bb-1f06-42fa-b941-37ba37e81db3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:27:44.473966 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6d60bb-1f06-42fa-b941-37ba37e81db3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:27:44.475374 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6d60bb-1f06-42fa-b941-37ba37e81db3 service neutron] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:27:44.488892 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1205/2435] 10.222.0.23 () {60 vars in 1365 bytes} [Thu May 19 12:27:44 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 255 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:27:48.630620 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c89272-387e-4b02-90ae-fb918ab47d43 None None] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:27:48.706921 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c89272-387e-4b02-90ae-fb918ab47d43 None None] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:27:48.712058 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09815d62-7175-41ea-b094-26dbbcafe16f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:27:48.712998 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09815d62-7175-41ea-b094-26dbbcafe16f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:27:48.713331 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09815d62-7175-41ea-b094-26dbbcafe16f admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:27:48.713587 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09815d62-7175-41ea-b094-26dbbcafe16f admin admin] PATH_INFO: `/v3/users/577e68f0e5fa48f49696e6a511698c50` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:27:48.714855 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09815d62-7175-41ea-b094-26dbbcafe16f admin admin] RBAC: Authorizing `identity:delete_user(user_id=577e68f0e5fa48f49696e6a511698c50)` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:27:48.716943 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09815d62-7175-41ea-b094-26dbbcafe16f admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:27:48.969532 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-09815d62-7175-41ea-b094-26dbbcafe16f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '577e68f0e5fa48f49696e6a511698c50'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:27:48.985815 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-09815d62-7175-41ea-b094-26dbbcafe16f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '577e68f0e5fa48f49696e6a511698c50'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:27:48.988937 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-09815d62-7175-41ea-b094-26dbbcafe16f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '577e68f0e5fa48f49696e6a511698c50'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:27:49.085169 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-09815d62-7175-41ea-b094-26dbbcafe16f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '577e68f0e5fa48f49696e6a511698c50'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:27:49.094383 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-09815d62-7175-41ea-b094-26dbbcafe16f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '577e68f0e5fa48f49696e6a511698c50'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:27:49.101213 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1231/2436] 10.222.0.23 () {58 vars in 1223 bytes} [Thu May 19 12:27:48 2022] DELETE /identity/v3/users/577e68f0e5fa48f49696e6a511698c50 => generated 0 bytes in 471 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:27:49.107996 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f25ad7c-734b-4f29-8b3e-470d8bd663e2 None None] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:27:49.108551 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f25ad7c-734b-4f29-8b3e-470d8bd663e2 None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:27:49.108551 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f25ad7c-734b-4f29-8b3e-470d8bd663e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:27:49.126773 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.auth.core [None req-2f25ad7c-734b-4f29-8b3e-470d8bd663e2 None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:27:49.128041 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-2f25ad7c-734b-4f29-8b3e-470d8bd663e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:27:49.191998 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1206/2437] 10.222.0.23 () {58 vars in 957 bytes} [Thu May 19 12:27:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:27:49.202338 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09815d62-7175-41ea-b094-26dbbcafe16f admin admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:27:49.204090 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-09815d62-7175-41ea-b094-26dbbcafe16f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:27:49.247894 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:27:49.280461 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09815d62-7175-41ea-b094-26dbbcafe16f admin admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:27:49.286480 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd4d306-6505-48b2-b58b-03a6cb064400 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a00b3fa7497456895e5e0bf105c4fc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd70d25bf69314820b8aaecb65576978a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:27:49.286480 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd4d306-6505-48b2-b58b-03a6cb064400 service neutron] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:27:49.286480 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd4d306-6505-48b2-b58b-03a6cb064400 service neutron] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:27:49.286480 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd4d306-6505-48b2-b58b-03a6cb064400 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:27:49.296403 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd4d306-6505-48b2-b58b-03a6cb064400 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:27:49.297882 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd4d306-6505-48b2-b58b-03a6cb064400 service neutron] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:27:49.308423 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1232/2438] 10.222.0.23 () {60 vars in 1365 bytes} [Thu May 19 12:27:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:27:49.762701 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f25ad7c-734b-4f29-8b3e-470d8bd663e2 None None] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:27:49.765175 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-2f25ad7c-734b-4f29-8b3e-470d8bd663e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:27:49.782012 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f25ad7c-734b-4f29-8b3e-470d8bd663e2 None None] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:27:49.789964 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec0ec42-2d27-43fc-a8f2-f7df4680f3d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:27:49.791135 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec0ec42-2d27-43fc-a8f2-f7df4680f3d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:27:49.791756 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec0ec42-2d27-43fc-a8f2-f7df4680f3d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:27:49.792177 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec0ec42-2d27-43fc-a8f2-f7df4680f3d8 admin admin] PATH_INFO: `/v3/projects/0a22db288abd45aba333be3640ead9ee` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:27:49.793726 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec0ec42-2d27-43fc-a8f2-f7df4680f3d8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0a22db288abd45aba333be3640ead9ee)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:27:49.796000 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec0ec42-2d27-43fc-a8f2-f7df4680f3d8 admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:27:49.996505 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-4ec0ec42-2d27-43fc-a8f2-f7df4680f3d8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0a22db288abd45aba333be3640ead9ee', 'request_id': 'req-4ec0ec42-2d27-43fc-a8f2-f7df4680f3d8'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:27:49.997681 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:27:50.003573 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-4ec0ec42-2d27-43fc-a8f2-f7df4680f3d8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0a22db288abd45aba333be3640ead9ee', 'request_id': 'req-4ec0ec42-2d27-43fc-a8f2-f7df4680f3d8'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:27:50.120595 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-4ec0ec42-2d27-43fc-a8f2-f7df4680f3d8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0a22db288abd45aba333be3640ead9ee', 'request_id': 'req-4ec0ec42-2d27-43fc-a8f2-f7df4680f3d8'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:27:50.149124 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-4ec0ec42-2d27-43fc-a8f2-f7df4680f3d8 admin admin] The token cache is being invalidate because project 0a22db288abd45aba333be3640ead9ee was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80539) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 12:27:50.149492 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-4ec0ec42-2d27-43fc-a8f2-f7df4680f3d8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:27:50.150263 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-4ec0ec42-2d27-43fc-a8f2-f7df4680f3d8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:27:50.157413 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1207/2439] 10.222.0.23 () {58 vars in 1232 bytes} [Thu May 19 12:27:49 2022] DELETE /identity/v3/projects/0a22db288abd45aba333be3640ead9ee => generated 0 bytes in 391 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:27:50.162803 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62c5665-00ba-49ba-832a-5dfb649a242a None None] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:27:50.163483 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62c5665-00ba-49ba-832a-5dfb649a242a None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:27:50.164415 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62c5665-00ba-49ba-832a-5dfb649a242a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:27:50.188710 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.auth.core [None req-a62c5665-00ba-49ba-832a-5dfb649a242a None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:27:50.190073 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-a62c5665-00ba-49ba-832a-5dfb649a242a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:27:50.281094 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1233/2440] 10.222.0.23 () {58 vars in 957 bytes} [Thu May 19 12:27:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:27:50.294548 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04909e9a-bbba-4adc-a0e2-eaf238ebcf90 None None] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:27:50.295006 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04909e9a-bbba-4adc-a0e2-eaf238ebcf90 None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:27:50.295560 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04909e9a-bbba-4adc-a0e2-eaf238ebcf90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:27:50.316566 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.auth.core [None req-04909e9a-bbba-4adc-a0e2-eaf238ebcf90 None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:27:50.317859 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-04909e9a-bbba-4adc-a0e2-eaf238ebcf90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:27:50.329307 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1208/2441] 10.222.0.23 () {58 vars in 957 bytes} [Thu May 19 12:27:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:27:50.334360 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62c5665-00ba-49ba-832a-5dfb649a242a None None] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:27:50.345994 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62c5665-00ba-49ba-832a-5dfb649a242a None None] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:27:50.350917 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299ee3fc-45fb-4d43-937e-eefc2f6d0340 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:27:50.351656 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299ee3fc-45fb-4d43-937e-eefc2f6d0340 admin admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:27:50.351979 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299ee3fc-45fb-4d43-937e-eefc2f6d0340 admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:27:50.352259 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299ee3fc-45fb-4d43-937e-eefc2f6d0340 admin admin] PATH_INFO: `/v3/domains` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:27:50.352999 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299ee3fc-45fb-4d43-937e-eefc2f6d0340 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:27:50.353409 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299ee3fc-45fb-4d43-937e-eefc2f6d0340 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:27:50.355909 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299ee3fc-45fb-4d43-937e-eefc2f6d0340 admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:27:50.373576 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1234/2442] 10.222.0.23 () {58 vars in 1152 bytes} [Thu May 19 12:27:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:27:50.377895 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04909e9a-bbba-4adc-a0e2-eaf238ebcf90 None None] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:27:50.389341 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04909e9a-bbba-4adc-a0e2-eaf238ebcf90 None None] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:27:50.394245 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f73879b-9be7-4c3c-824d-9ca22ee7f3d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:27:50.394918 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f73879b-9be7-4c3c-824d-9ca22ee7f3d1 admin admin] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:27:50.395213 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f73879b-9be7-4c3c-824d-9ca22ee7f3d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:27:50.395488 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f73879b-9be7-4c3c-824d-9ca22ee7f3d1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:27:50.396237 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f73879b-9be7-4c3c-824d-9ca22ee7f3d1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:27:50.398300 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f73879b-9be7-4c3c-824d-9ca22ee7f3d1 admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:27:50.494178 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1209/2443] 10.222.0.23 () {60 vars in 1152 bytes} [Thu May 19 12:27:50 2022] POST /identity/v3/projects => generated 365 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 12:27:50.499931 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299ee3fc-45fb-4d43-937e-eefc2f6d0340 admin admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:27:50.511091 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299ee3fc-45fb-4d43-937e-eefc2f6d0340 admin admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:27:50.585698 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3eb600-94cb-4444-9086-4b791d9ffb92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:27:50.585698 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3eb600-94cb-4444-9086-4b791d9ffb92 admin admin] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:27:50.585698 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3eb600-94cb-4444-9086-4b791d9ffb92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:27:50.585698 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3eb600-94cb-4444-9086-4b791d9ffb92 admin admin] PATH_INFO: `/v3/users` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:27:50.586344 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3eb600-94cb-4444-9086-4b791d9ffb92 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:27:50.586344 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3eb600-94cb-4444-9086-4b791d9ffb92 admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:27:50.700378 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1235/2444] 10.222.0.23 () {60 vars in 1143 bytes} [Thu May 19 12:27:50 2022] POST /identity/v3/users => generated 283 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 12:27:50.706055 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f73879b-9be7-4c3c-824d-9ca22ee7f3d1 admin admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:27:50.716104 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f73879b-9be7-4c3c-824d-9ca22ee7f3d1 admin admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:27:50.723274 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2a93e7-dfb2-4918-8cbd-d292a89ab1d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:27:50.724014 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2a93e7-dfb2-4918-8cbd-d292a89ab1d7 admin admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:27:50.725120 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2a93e7-dfb2-4918-8cbd-d292a89ab1d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:27:50.725120 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2a93e7-dfb2-4918-8cbd-d292a89ab1d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:27:50.725567 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2a93e7-dfb2-4918-8cbd-d292a89ab1d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:27:50.727513 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2a93e7-dfb2-4918-8cbd-d292a89ab1d7 admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:27:50.743689 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1210/2445] 10.222.0.23 () {58 vars in 1121 bytes} [Thu May 19 12:27:50 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 12:27:50.748213 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3eb600-94cb-4444-9086-4b791d9ffb92 admin admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:27:50.759496 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3eb600-94cb-4444-9086-4b791d9ffb92 admin admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:27:50.765023 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0c742b-3f9a-4517-9e9c-2d1d4d88fab5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:27:50.765745 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0c742b-3f9a-4517-9e9c-2d1d4d88fab5 admin admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:27:50.766035 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0c742b-3f9a-4517-9e9c-2d1d4d88fab5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:27:50.766313 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0c742b-3f9a-4517-9e9c-2d1d4d88fab5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:27:50.767057 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0c742b-3f9a-4517-9e9c-2d1d4d88fab5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:27:50.768747 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0c742b-3f9a-4517-9e9c-2d1d4d88fab5 admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:27:50.781413 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1236/2446] 10.222.0.23 () {58 vars in 1121 bytes} [Thu May 19 12:27:50 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 12:27:50.786898 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2a93e7-dfb2-4918-8cbd-d292a89ab1d7 admin admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:27:50.798574 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2a93e7-dfb2-4918-8cbd-d292a89ab1d7 admin admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:27:50.804836 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d86b44-7102-4f26-a816-50f2fad96de6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:27:50.808418 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d86b44-7102-4f26-a816-50f2fad96de6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:27:50.808418 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d86b44-7102-4f26-a816-50f2fad96de6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:27:50.808418 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d86b44-7102-4f26-a816-50f2fad96de6 admin admin] PATH_INFO: `/v3/projects/004b5089fa3a466ba425b9df553b1167/users/340e0689c4b6404cbb9aa10a3824b679/roles/259eb400cdb4439db72d14456ae9cd08` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:27:50.826720 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d86b44-7102-4f26-a816-50f2fad96de6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=004b5089fa3a466ba425b9df553b1167, user_id=340e0689c4b6404cbb9aa10a3824b679, role_id=259eb400cdb4439db72d14456ae9cd08)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:27:50.830049 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d86b44-7102-4f26-a816-50f2fad96de6 admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:27:50.943863 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1211/2447] 10.222.0.23 () {60 vars in 1482 bytes} [Thu May 19 12:27:50 2022] PUT /identity/v3/projects/004b5089fa3a466ba425b9df553b1167/users/340e0689c4b6404cbb9aa10a3824b679/roles/259eb400cdb4439db72d14456ae9cd08 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:27:50.955490 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc8981e-9a37-4ea6-97e5-bbb2229faceb None None] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:27:50.955838 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc8981e-9a37-4ea6-97e5-bbb2229faceb None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:27:50.956117 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc8981e-9a37-4ea6-97e5-bbb2229faceb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:27:50.988298 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.auth.core [None req-5fc8981e-9a37-4ea6-97e5-bbb2229faceb None None] MFA Rules not processed for user `340e0689c4b6404cbb9aa10a3824b679`. Rule list: `[]` (Enabled: `True`). {{(pid=80540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:27:50.989584 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-5fc8981e-9a37-4ea6-97e5-bbb2229faceb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:27:51.014289 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:27:51.072859 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1237/2448] 10.222.0.23 () {58 vars in 957 bytes} [Thu May 19 12:27:50 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:27:51.085674 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d86b44-7102-4f26-a816-50f2fad96de6 admin admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:27:51.091050 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-47d86b44-7102-4f26-a816-50f2fad96de6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:27:51.170082 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d86b44-7102-4f26-a816-50f2fad96de6 admin admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:27:51.176013 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e92a7b-62ed-4124-a40f-3d62535a4820 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8800254c0e784e42a791fb9abc274d79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd70d25bf69314820b8aaecb65576978a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:27:51.176736 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e92a7b-62ed-4124-a40f-3d62535a4820 service cinder] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:27:51.177015 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e92a7b-62ed-4124-a40f-3d62535a4820 service cinder] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:27:51.177293 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e92a7b-62ed-4124-a40f-3d62535a4820 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:27:51.187539 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e92a7b-62ed-4124-a40f-3d62535a4820 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:27:51.189233 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e92a7b-62ed-4124-a40f-3d62535a4820 service cinder] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:27:51.199642 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1212/2449] 10.222.0.23 () {60 vars in 1365 bytes} [Thu May 19 12:27:51 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 12:27:51.942398 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc8981e-9a37-4ea6-97e5-bbb2229faceb None None] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:27:51.952486 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-5fc8981e-9a37-4ea6-97e5-bbb2229faceb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:27:52.034198 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc8981e-9a37-4ea6-97e5-bbb2229faceb None None] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:27:52.037902 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6519bbaa-caed-4f82-b7ea-70dbdcbcc7d0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a00b3fa7497456895e5e0bf105c4fc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd70d25bf69314820b8aaecb65576978a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:27:52.038595 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6519bbaa-caed-4f82-b7ea-70dbdcbcc7d0 service neutron] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:27:52.038892 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6519bbaa-caed-4f82-b7ea-70dbdcbcc7d0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:27:52.039174 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6519bbaa-caed-4f82-b7ea-70dbdcbcc7d0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:27:52.041060 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-6519bbaa-caed-4f82-b7ea-70dbdcbcc7d0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:27:52.127062 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6519bbaa-caed-4f82-b7ea-70dbdcbcc7d0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:27:52.128561 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6519bbaa-caed-4f82-b7ea-70dbdcbcc7d0 service neutron] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:27:52.140555 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1238/2450] 10.222.0.23 () {60 vars in 1365 bytes} [Thu May 19 12:27:51 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 199 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:27:52.516130 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e92a7b-62ed-4124-a40f-3d62535a4820 service cinder] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:27:52.528247 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e92a7b-62ed-4124-a40f-3d62535a4820 service cinder] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:27:52.532487 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8c288c-d62d-45af-9da7-615cd486852a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a00b3fa7497456895e5e0bf105c4fc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd70d25bf69314820b8aaecb65576978a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:27:52.533142 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8c288c-d62d-45af-9da7-615cd486852a service neutron] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:27:52.533430 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8c288c-d62d-45af-9da7-615cd486852a service neutron] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:27:52.533710 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8c288c-d62d-45af-9da7-615cd486852a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:27:52.535570 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-9a8c288c-d62d-45af-9da7-615cd486852a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:27:52.550216 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8c288c-d62d-45af-9da7-615cd486852a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:27:52.551679 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8c288c-d62d-45af-9da7-615cd486852a service neutron] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:27:52.564065 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1213/2451] 10.222.0.23 () {60 vars in 1365 bytes} [Thu May 19 12:27:52 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:27:57.204053 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6519bbaa-caed-4f82-b7ea-70dbdcbcc7d0 service neutron] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:27:57.206145 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-6519bbaa-caed-4f82-b7ea-70dbdcbcc7d0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:27:57.217523 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6519bbaa-caed-4f82-b7ea-70dbdcbcc7d0 service neutron] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:27:57.222186 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f999128f-f170-49b3-aaea-5b6950e3398c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a00b3fa7497456895e5e0bf105c4fc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd70d25bf69314820b8aaecb65576978a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:27:57.222850 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f999128f-f170-49b3-aaea-5b6950e3398c service neutron] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:27:57.223138 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f999128f-f170-49b3-aaea-5b6950e3398c service neutron] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:27:57.223425 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f999128f-f170-49b3-aaea-5b6950e3398c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:27:57.227828 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-f999128f-f170-49b3-aaea-5b6950e3398c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:27:57.295233 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f999128f-f170-49b3-aaea-5b6950e3398c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:27:57.296670 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f999128f-f170-49b3-aaea-5b6950e3398c service neutron] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:27:57.301243 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:27:57.328333 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1239/2452] 10.222.0.23 () {60 vars in 1365 bytes} [Thu May 19 12:27:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:27:58.843327 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8c288c-d62d-45af-9da7-615cd486852a service neutron] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:27:58.853365 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8c288c-d62d-45af-9da7-615cd486852a service neutron] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:27:58.857868 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0960c33-b82b-411c-828f-97ddb24bad47 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a00b3fa7497456895e5e0bf105c4fc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd70d25bf69314820b8aaecb65576978a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:27:58.858569 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0960c33-b82b-411c-828f-97ddb24bad47 service neutron] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:27:58.858865 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0960c33-b82b-411c-828f-97ddb24bad47 service neutron] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:27:58.859143 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0960c33-b82b-411c-828f-97ddb24bad47 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:27:58.861408 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-e0960c33-b82b-411c-828f-97ddb24bad47 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:27:58.934837 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0960c33-b82b-411c-828f-97ddb24bad47 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:27:58.936418 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0960c33-b82b-411c-828f-97ddb24bad47 service neutron] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:27:58.940758 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=94,keystone:INSERT=3,keystone:DELETE=3 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:27:58.947618 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1214/2453] 10.222.0.23 () {60 vars in 1365 bytes} [Thu May 19 12:27:58 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:28:06.937214 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f999128f-f170-49b3-aaea-5b6950e3398c service neutron] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:28:06.938849 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-f999128f-f170-49b3-aaea-5b6950e3398c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:28:06.952771 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f999128f-f170-49b3-aaea-5b6950e3398c service neutron] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:28:06.957735 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8069580a-3fe6-4ca2-b084-1e739e593ceb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:28:06.958629 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8069580a-3fe6-4ca2-b084-1e739e593ceb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:28:06.958914 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8069580a-3fe6-4ca2-b084-1e739e593ceb admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:28:06.959195 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8069580a-3fe6-4ca2-b084-1e739e593ceb admin admin] PATH_INFO: `/v3/users/3124804c65204e9e8dc4c26e572390b9` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:28:06.960421 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8069580a-3fe6-4ca2-b084-1e739e593ceb admin admin] RBAC: Authorizing `identity:delete_user(user_id=3124804c65204e9e8dc4c26e572390b9)` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:28:06.962380 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8069580a-3fe6-4ca2-b084-1e739e593ceb admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:28:07.201689 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-8069580a-3fe6-4ca2-b084-1e739e593ceb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3124804c65204e9e8dc4c26e572390b9'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:28:07.214516 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-8069580a-3fe6-4ca2-b084-1e739e593ceb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3124804c65204e9e8dc4c26e572390b9'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:28:07.215231 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-8069580a-3fe6-4ca2-b084-1e739e593ceb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3124804c65204e9e8dc4c26e572390b9'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:28:07.301617 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-8069580a-3fe6-4ca2-b084-1e739e593ceb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3124804c65204e9e8dc4c26e572390b9'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:28:07.309347 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-8069580a-3fe6-4ca2-b084-1e739e593ceb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3124804c65204e9e8dc4c26e572390b9'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:28:07.315613 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1240/2454] 10.222.0.23 () {58 vars in 1223 bytes} [Thu May 19 12:28:06 2022] DELETE /identity/v3/users/3124804c65204e9e8dc4c26e572390b9 => generated 0 bytes in 379 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:28:07.584152 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0960c33-b82b-411c-828f-97ddb24bad47 service neutron] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:28:07.595649 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0960c33-b82b-411c-828f-97ddb24bad47 service neutron] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:28:07.656720 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f7861b-bd00-4077-91ba-f112477d12da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:28:07.657598 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f7861b-bd00-4077-91ba-f112477d12da admin admin] REQUEST_METHOD: `DELETE` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:28:07.657887 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f7861b-bd00-4077-91ba-f112477d12da admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:28:07.658166 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f7861b-bd00-4077-91ba-f112477d12da admin admin] PATH_INFO: `/v3/projects/0beba4cdb9854497b6cedd3e76059e12` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:28:07.659410 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f7861b-bd00-4077-91ba-f112477d12da admin admin] RBAC: Authorizing `identity:delete_project(project_id=0beba4cdb9854497b6cedd3e76059e12)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:28:07.661384 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f7861b-bd00-4077-91ba-f112477d12da admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:28:07.787912 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-45f7861b-bd00-4077-91ba-f112477d12da admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0beba4cdb9854497b6cedd3e76059e12', 'request_id': 'req-45f7861b-bd00-4077-91ba-f112477d12da'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:28:07.795522 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-45f7861b-bd00-4077-91ba-f112477d12da admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0beba4cdb9854497b6cedd3e76059e12', 'request_id': 'req-45f7861b-bd00-4077-91ba-f112477d12da'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:28:07.862491 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-45f7861b-bd00-4077-91ba-f112477d12da admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0beba4cdb9854497b6cedd3e76059e12', 'request_id': 'req-45f7861b-bd00-4077-91ba-f112477d12da'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:28:07.873573 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-45f7861b-bd00-4077-91ba-f112477d12da admin admin] The token cache is being invalidate because project 0beba4cdb9854497b6cedd3e76059e12 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80539) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 12:28:07.873777 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-45f7861b-bd00-4077-91ba-f112477d12da admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:28:07.874413 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-45f7861b-bd00-4077-91ba-f112477d12da admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:28:07.875984 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1215/2455] 10.222.0.23 () {58 vars in 1232 bytes} [Thu May 19 12:28:07 2022] DELETE /identity/v3/projects/0beba4cdb9854497b6cedd3e76059e12 => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:28:07.880177 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8069580a-3fe6-4ca2-b084-1e739e593ceb admin admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:28:07.881554 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-8069580a-3fe6-4ca2-b084-1e739e593ceb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:28:07.895586 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8069580a-3fe6-4ca2-b084-1e739e593ceb admin admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:28:07.961064 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef20211-9517-433c-a366-26155353175a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:28:07.961950 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef20211-9517-433c-a366-26155353175a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:28:07.962240 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef20211-9517-433c-a366-26155353175a admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:28:07.962526 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef20211-9517-433c-a366-26155353175a admin admin] PATH_INFO: `/v3/users/82b6198e186449c5bae4c6c41ed2d813` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:28:07.964633 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef20211-9517-433c-a366-26155353175a admin admin] RBAC: Authorizing `identity:delete_user(user_id=82b6198e186449c5bae4c6c41ed2d813)` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:28:07.966615 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef20211-9517-433c-a366-26155353175a admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:28:08.185245 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-1ef20211-9517-433c-a366-26155353175a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '82b6198e186449c5bae4c6c41ed2d813'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:28:08.197640 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-1ef20211-9517-433c-a366-26155353175a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '82b6198e186449c5bae4c6c41ed2d813'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:28:08.198560 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-1ef20211-9517-433c-a366-26155353175a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '82b6198e186449c5bae4c6c41ed2d813'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:28:08.252150 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-1ef20211-9517-433c-a366-26155353175a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '82b6198e186449c5bae4c6c41ed2d813'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:28:08.253862 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:28:08.263135 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-1ef20211-9517-433c-a366-26155353175a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '82b6198e186449c5bae4c6c41ed2d813'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:28:08.270566 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1241/2456] 10.222.0.23 () {58 vars in 1223 bytes} [Thu May 19 12:28:07 2022] DELETE /identity/v3/users/82b6198e186449c5bae4c6c41ed2d813 => generated 0 bytes in 391 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:28:08.556350 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f7861b-bd00-4077-91ba-f112477d12da admin admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:28:08.558487 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-45f7861b-bd00-4077-91ba-f112477d12da admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:28:08.571558 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f7861b-bd00-4077-91ba-f112477d12da admin admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:28:08.636785 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bfac0e-b733-4988-b599-836f200aebdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:28:08.637659 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bfac0e-b733-4988-b599-836f200aebdc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:28:08.637945 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bfac0e-b733-4988-b599-836f200aebdc admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:28:08.638219 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bfac0e-b733-4988-b599-836f200aebdc admin admin] PATH_INFO: `/v3/projects/03e87054b75946338efbef7540c97760` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:28:08.639453 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79bfac0e-b733-4988-b599-836f200aebdc admin admin] RBAC: Authorizing `identity:delete_project(project_id=03e87054b75946338efbef7540c97760)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:28:08.641409 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79bfac0e-b733-4988-b599-836f200aebdc admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:28:08.748649 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:28:08.762350 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-79bfac0e-b733-4988-b599-836f200aebdc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '03e87054b75946338efbef7540c97760', 'request_id': 'req-79bfac0e-b733-4988-b599-836f200aebdc'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:28:08.768803 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-79bfac0e-b733-4988-b599-836f200aebdc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '03e87054b75946338efbef7540c97760', 'request_id': 'req-79bfac0e-b733-4988-b599-836f200aebdc'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:28:08.896723 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-79bfac0e-b733-4988-b599-836f200aebdc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '03e87054b75946338efbef7540c97760', 'request_id': 'req-79bfac0e-b733-4988-b599-836f200aebdc'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:28:08.908008 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-79bfac0e-b733-4988-b599-836f200aebdc admin admin] The token cache is being invalidate because project 03e87054b75946338efbef7540c97760 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80539) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 12:28:08.908294 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-79bfac0e-b733-4988-b599-836f200aebdc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:28:08.908920 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-79bfac0e-b733-4988-b599-836f200aebdc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:28:08.910447 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1216/2457] 10.222.0.23 () {58 vars in 1232 bytes} [Thu May 19 12:28:08 2022] DELETE /identity/v3/projects/03e87054b75946338efbef7540c97760 => generated 0 bytes in 355 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:28:18.393099 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=5 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:28:18.970006 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=13,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:28:35.342759 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9e1610-9bbe-4aa3-a401-caafb6c0f641 None None] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:28:35.343325 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9e1610-9bbe-4aa3-a401-caafb6c0f641 None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:28:35.343325 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9e1610-9bbe-4aa3-a401-caafb6c0f641 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:28:35.366373 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.auth.core [None req-de9e1610-9bbe-4aa3-a401-caafb6c0f641 None None] MFA Rules not processed for user `0c3181fb4f584a549a786adfefdf3dfb`. Rule list: `[]` (Enabled: `True`). {{(pid=80540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:28:35.367691 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-de9e1610-9bbe-4aa3-a401-caafb6c0f641 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:28:35.441931 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1242/2458] 10.222.0.23 () {58 vars in 1009 bytes} [Thu May 19 12:28:35 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:28:35.781203 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bfac0e-b733-4988-b599-836f200aebdc admin admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:28:35.782688 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-79bfac0e-b733-4988-b599-836f200aebdc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:28:35.853655 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bfac0e-b733-4988-b599-836f200aebdc admin admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:28:35.859915 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d9215a-49c2-4ffc-aea9-9562b8934c16 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15cbfc73ae1243d2a8b1e70f9cd99876', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd70d25bf69314820b8aaecb65576978a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:28:35.860861 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d9215a-49c2-4ffc-aea9-9562b8934c16 service swift] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:28:35.861151 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d9215a-49c2-4ffc-aea9-9562b8934c16 service swift] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:28:35.861428 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d9215a-49c2-4ffc-aea9-9562b8934c16 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:28:35.870380 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d9215a-49c2-4ffc-aea9-9562b8934c16 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:28:35.872244 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d9215a-49c2-4ffc-aea9-9562b8934c16 service swift] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:28:35.880818 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1217/2459] 10.222.0.23 () {60 vars in 1384 bytes} [Thu May 19 12:28:35 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 19 12:28:45.435386 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=25 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:28:45.877181 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=32 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:28:47.763009 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9e1610-9bbe-4aa3-a401-caafb6c0f641 None None] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:28:47.765106 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-de9e1610-9bbe-4aa3-a401-caafb6c0f641 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:28:47.902490 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9e1610-9bbe-4aa3-a401-caafb6c0f641 None None] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:28:47.907491 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d548cd-b390-4d4e-a016-a81ccf9a735d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8800254c0e784e42a791fb9abc274d79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd70d25bf69314820b8aaecb65576978a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:28:47.908604 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d548cd-b390-4d4e-a016-a81ccf9a735d service cinder] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:28:47.909130 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d548cd-b390-4d4e-a016-a81ccf9a735d service cinder] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:28:47.909576 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d548cd-b390-4d4e-a016-a81ccf9a735d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:28:47.913067 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-48d548cd-b390-4d4e-a016-a81ccf9a735d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:28:48.102626 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d548cd-b390-4d4e-a016-a81ccf9a735d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:28:48.107312 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d548cd-b390-4d4e-a016-a81ccf9a735d service cinder] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:28:48.162424 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1243/2460] 10.222.0.23 () {60 vars in 1365 bytes} [Thu May 19 12:28:47 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:28:51.687771 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d9215a-49c2-4ffc-aea9-9562b8934c16 service swift] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:28:51.689650 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-c0d9215a-49c2-4ffc-aea9-9562b8934c16 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:28:51.770880 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d9215a-49c2-4ffc-aea9-9562b8934c16 service swift] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:28:51.774691 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa5eb87-d03e-43ff-82c3-61056296f42f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a00b3fa7497456895e5e0bf105c4fc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd70d25bf69314820b8aaecb65576978a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:28:51.775350 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa5eb87-d03e-43ff-82c3-61056296f42f service neutron] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:28:51.775630 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa5eb87-d03e-43ff-82c3-61056296f42f service neutron] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:28:51.775944 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa5eb87-d03e-43ff-82c3-61056296f42f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:28:51.778119 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-caa5eb87-d03e-43ff-82c3-61056296f42f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:28:51.859372 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa5eb87-d03e-43ff-82c3-61056296f42f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:28:51.867332 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa5eb87-d03e-43ff-82c3-61056296f42f service neutron] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:28:51.872405 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1218/2461] 10.222.0.23 () {60 vars in 1365 bytes} [Thu May 19 12:28:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:28:53.497184 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d548cd-b390-4d4e-a016-a81ccf9a735d service cinder] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:28:53.507399 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d548cd-b390-4d4e-a016-a81ccf9a735d service cinder] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:28:53.511907 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c261408d-6265-444c-bc75-cda209e93859 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a00b3fa7497456895e5e0bf105c4fc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd70d25bf69314820b8aaecb65576978a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:28:53.512585 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c261408d-6265-444c-bc75-cda209e93859 service neutron] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:28:53.512874 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c261408d-6265-444c-bc75-cda209e93859 service neutron] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:28:53.513153 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c261408d-6265-444c-bc75-cda209e93859 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:28:53.515165 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-c261408d-6265-444c-bc75-cda209e93859 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:28:53.526241 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c261408d-6265-444c-bc75-cda209e93859 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:28:53.528674 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c261408d-6265-444c-bc75-cda209e93859 service neutron] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:28:53.538255 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1244/2462] 10.222.0.23 () {60 vars in 1365 bytes} [Thu May 19 12:28:53 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:28:54.475873 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa5eb87-d03e-43ff-82c3-61056296f42f service neutron] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:28:54.477510 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-caa5eb87-d03e-43ff-82c3-61056296f42f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:28:54.498802 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa5eb87-d03e-43ff-82c3-61056296f42f service neutron] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:28:54.504717 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54eb0bdc-91ac-4c80-9310-6389d31dfdc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:28:54.505784 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54eb0bdc-91ac-4c80-9310-6389d31dfdc9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:28:54.506045 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54eb0bdc-91ac-4c80-9310-6389d31dfdc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:28:54.506329 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54eb0bdc-91ac-4c80-9310-6389d31dfdc9 admin admin] PATH_INFO: `/v3/users/340e0689c4b6404cbb9aa10a3824b679` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:28:54.508005 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54eb0bdc-91ac-4c80-9310-6389d31dfdc9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=340e0689c4b6404cbb9aa10a3824b679)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:28:54.510217 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54eb0bdc-91ac-4c80-9310-6389d31dfdc9 admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:28:54.837888 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-54eb0bdc-91ac-4c80-9310-6389d31dfdc9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '340e0689c4b6404cbb9aa10a3824b679'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:28:54.861141 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-54eb0bdc-91ac-4c80-9310-6389d31dfdc9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '340e0689c4b6404cbb9aa10a3824b679'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:28:54.862713 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-54eb0bdc-91ac-4c80-9310-6389d31dfdc9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '340e0689c4b6404cbb9aa10a3824b679'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:28:55.067506 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-54eb0bdc-91ac-4c80-9310-6389d31dfdc9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '340e0689c4b6404cbb9aa10a3824b679'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:28:55.078948 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-54eb0bdc-91ac-4c80-9310-6389d31dfdc9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '340e0689c4b6404cbb9aa10a3824b679'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:28:55.090284 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1219/2463] 10.222.0.23 () {58 vars in 1223 bytes} [Thu May 19 12:28:54 2022] DELETE /identity/v3/users/340e0689c4b6404cbb9aa10a3824b679 => generated 0 bytes in 615 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:28:55.096259 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe738f79-302b-40e3-987e-9dce578d6b28 None None] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:28:55.097142 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe738f79-302b-40e3-987e-9dce578d6b28 None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:28:55.097302 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe738f79-302b-40e3-987e-9dce578d6b28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:28:55.129704 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.auth.core [None req-fe738f79-302b-40e3-987e-9dce578d6b28 None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:28:55.131928 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-fe738f79-302b-40e3-987e-9dce578d6b28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:28:55.255002 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1245/2464] 10.222.0.23 () {58 vars in 957 bytes} [Thu May 19 12:28:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:28:55.267673 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54eb0bdc-91ac-4c80-9310-6389d31dfdc9 admin admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:28:55.269240 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-54eb0bdc-91ac-4c80-9310-6389d31dfdc9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:28:55.310301 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:28:55.361989 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54eb0bdc-91ac-4c80-9310-6389d31dfdc9 admin admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:28:55.371453 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4418e332-f39f-4ff2-b7ec-de5552fa59d9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a00b3fa7497456895e5e0bf105c4fc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd70d25bf69314820b8aaecb65576978a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:28:55.373818 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4418e332-f39f-4ff2-b7ec-de5552fa59d9 service neutron] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:28:55.373818 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4418e332-f39f-4ff2-b7ec-de5552fa59d9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:28:55.374175 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4418e332-f39f-4ff2-b7ec-de5552fa59d9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:28:55.393242 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4418e332-f39f-4ff2-b7ec-de5552fa59d9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:28:55.395810 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4418e332-f39f-4ff2-b7ec-de5552fa59d9 service neutron] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:28:55.415284 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1220/2465] 10.222.0.23 () {60 vars in 1365 bytes} [Thu May 19 12:28:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:28:56.295778 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe738f79-302b-40e3-987e-9dce578d6b28 None None] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:28:56.305675 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-fe738f79-302b-40e3-987e-9dce578d6b28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:28:56.328020 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe738f79-302b-40e3-987e-9dce578d6b28 None None] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:28:56.333940 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4475e880-0574-45f5-82e1-28b518cc7962 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:28:56.335016 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4475e880-0574-45f5-82e1-28b518cc7962 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:28:56.335302 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4475e880-0574-45f5-82e1-28b518cc7962 admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:28:56.335580 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4475e880-0574-45f5-82e1-28b518cc7962 admin admin] PATH_INFO: `/v3/projects/004b5089fa3a466ba425b9df553b1167` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:28:56.337101 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4475e880-0574-45f5-82e1-28b518cc7962 admin admin] RBAC: Authorizing `identity:delete_project(project_id=004b5089fa3a466ba425b9df553b1167)` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:28:56.339177 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4475e880-0574-45f5-82e1-28b518cc7962 admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:28:56.364886 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:28:56.603192 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-4475e880-0574-45f5-82e1-28b518cc7962 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '004b5089fa3a466ba425b9df553b1167', 'request_id': 'req-4475e880-0574-45f5-82e1-28b518cc7962'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:28:56.616814 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-4475e880-0574-45f5-82e1-28b518cc7962 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '004b5089fa3a466ba425b9df553b1167', 'request_id': 'req-4475e880-0574-45f5-82e1-28b518cc7962'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:28:56.629185 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-4475e880-0574-45f5-82e1-28b518cc7962 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '004b5089fa3a466ba425b9df553b1167', 'request_id': 'req-4475e880-0574-45f5-82e1-28b518cc7962'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:28:56.803083 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-4475e880-0574-45f5-82e1-28b518cc7962 admin admin] The token cache is being invalidate because project 004b5089fa3a466ba425b9df553b1167 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80540) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 12:28:56.803554 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-4475e880-0574-45f5-82e1-28b518cc7962 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:28:56.804813 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-4475e880-0574-45f5-82e1-28b518cc7962 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:28:56.806908 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1246/2466] 10.222.0.23 () {58 vars in 1232 bytes} [Thu May 19 12:28:56 2022] DELETE /identity/v3/projects/004b5089fa3a466ba425b9df553b1167 => generated 0 bytes in 512 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:28:57.400693 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4418e332-f39f-4ff2-b7ec-de5552fa59d9 service neutron] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:28:57.404443 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-4418e332-f39f-4ff2-b7ec-de5552fa59d9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:28:57.421277 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4418e332-f39f-4ff2-b7ec-de5552fa59d9 service neutron] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:28:57.496185 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb067bbc-ee99-455a-985d-6a422aa2fc99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:28:57.497317 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb067bbc-ee99-455a-985d-6a422aa2fc99 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:28:57.497661 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb067bbc-ee99-455a-985d-6a422aa2fc99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:28:57.498013 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb067bbc-ee99-455a-985d-6a422aa2fc99 admin admin] PATH_INFO: `/v3/users/9d762cce00ae4e468c358b0437f35deb` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:28:57.499689 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb067bbc-ee99-455a-985d-6a422aa2fc99 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9d762cce00ae4e468c358b0437f35deb)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:28:57.501747 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb067bbc-ee99-455a-985d-6a422aa2fc99 admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:28:57.861139 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-cb067bbc-ee99-455a-985d-6a422aa2fc99 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9d762cce00ae4e468c358b0437f35deb'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:28:57.890574 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-cb067bbc-ee99-455a-985d-6a422aa2fc99 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9d762cce00ae4e468c358b0437f35deb'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:28:57.891465 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-cb067bbc-ee99-455a-985d-6a422aa2fc99 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9d762cce00ae4e468c358b0437f35deb'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:28:58.061179 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-cb067bbc-ee99-455a-985d-6a422aa2fc99 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9d762cce00ae4e468c358b0437f35deb'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:28:58.070609 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-cb067bbc-ee99-455a-985d-6a422aa2fc99 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9d762cce00ae4e468c358b0437f35deb'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:28:58.078423 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1221/2467] 10.222.0.23 () {58 vars in 1223 bytes} [Thu May 19 12:28:57 2022] DELETE /identity/v3/users/9d762cce00ae4e468c358b0437f35deb => generated 0 bytes in 678 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:28:58.442622 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4475e880-0574-45f5-82e1-28b518cc7962 admin admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:28:58.445554 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-4475e880-0574-45f5-82e1-28b518cc7962 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:28:58.468576 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4475e880-0574-45f5-82e1-28b518cc7962 admin admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:28:58.568126 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebe8097-01a4-43cf-a08f-5d7166eee033 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:28:58.569767 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebe8097-01a4-43cf-a08f-5d7166eee033 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:28:58.570289 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebe8097-01a4-43cf-a08f-5d7166eee033 admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:28:58.570807 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebe8097-01a4-43cf-a08f-5d7166eee033 admin admin] PATH_INFO: `/v3/projects/9eafee25cfb14ca2bd5c10fe41a01644` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:28:58.573126 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ebe8097-01a4-43cf-a08f-5d7166eee033 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9eafee25cfb14ca2bd5c10fe41a01644)` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:28:58.577763 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ebe8097-01a4-43cf-a08f-5d7166eee033 admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:28:58.837130 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-7ebe8097-01a4-43cf-a08f-5d7166eee033 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9eafee25cfb14ca2bd5c10fe41a01644', 'request_id': 'req-7ebe8097-01a4-43cf-a08f-5d7166eee033'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:28:58.844586 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-7ebe8097-01a4-43cf-a08f-5d7166eee033 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9eafee25cfb14ca2bd5c10fe41a01644', 'request_id': 'req-7ebe8097-01a4-43cf-a08f-5d7166eee033'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:28:58.851982 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-7ebe8097-01a4-43cf-a08f-5d7166eee033 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9eafee25cfb14ca2bd5c10fe41a01644', 'request_id': 'req-7ebe8097-01a4-43cf-a08f-5d7166eee033'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:28:59.039003 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-7ebe8097-01a4-43cf-a08f-5d7166eee033 admin admin] The token cache is being invalidate because project 9eafee25cfb14ca2bd5c10fe41a01644 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80540) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 12:28:59.039406 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-7ebe8097-01a4-43cf-a08f-5d7166eee033 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:28:59.040221 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-7ebe8097-01a4-43cf-a08f-5d7166eee033 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:28:59.042459 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1247/2468] 10.222.0.23 () {58 vars in 1232 bytes} [Thu May 19 12:28:58 2022] DELETE /identity/v3/projects/9eafee25cfb14ca2bd5c10fe41a01644 => generated 0 bytes in 600 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:28:59.047846 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ab0eb0-6d08-4625-a5b1-089409e01b6f None None] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:28:59.048578 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ab0eb0-6d08-4625-a5b1-089409e01b6f None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:28:59.048578 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ab0eb0-6d08-4625-a5b1-089409e01b6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:28:59.070320 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.auth.core [None req-b7ab0eb0-6d08-4625-a5b1-089409e01b6f None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:28:59.071627 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-b7ab0eb0-6d08-4625-a5b1-089409e01b6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:28:59.140646 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:28:59.161896 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1222/2469] 10.222.0.23 () {58 vars in 957 bytes} [Thu May 19 12:28:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:28:59.172720 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2c905b-8992-46c9-b651-20c28cd80265 None None] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:28:59.172720 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2c905b-8992-46c9-b651-20c28cd80265 None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:28:59.172720 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2c905b-8992-46c9-b651-20c28cd80265 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:28:59.194795 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.auth.core [None req-3d2c905b-8992-46c9-b651-20c28cd80265 None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:28:59.196192 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-3d2c905b-8992-46c9-b651-20c28cd80265 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:28:59.206123 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1248/2470] 10.222.0.23 () {58 vars in 957 bytes} [Thu May 19 12:28:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:28:59.211863 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ab0eb0-6d08-4625-a5b1-089409e01b6f None None] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:28:59.229809 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ab0eb0-6d08-4625-a5b1-089409e01b6f None None] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:28:59.237595 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337df783-6dba-495d-8d32-cf2dd44eec1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:28:59.238621 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337df783-6dba-495d-8d32-cf2dd44eec1e admin admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:28:59.239041 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337df783-6dba-495d-8d32-cf2dd44eec1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:28:59.239466 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337df783-6dba-495d-8d32-cf2dd44eec1e admin admin] PATH_INFO: `/v3/domains` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:28:59.240692 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337df783-6dba-495d-8d32-cf2dd44eec1e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:28:59.241176 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337df783-6dba-495d-8d32-cf2dd44eec1e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:28:59.244085 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337df783-6dba-495d-8d32-cf2dd44eec1e admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:28:59.265055 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1223/2471] 10.222.0.23 () {58 vars in 1152 bytes} [Thu May 19 12:28:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:28:59.274574 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2c905b-8992-46c9-b651-20c28cd80265 None None] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:28:59.285893 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2c905b-8992-46c9-b651-20c28cd80265 None None] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:28:59.291046 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f6a4f8-f8d7-4d7c-82de-a3d9a4affe2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:28:59.291734 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f6a4f8-f8d7-4d7c-82de-a3d9a4affe2a admin admin] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:28:59.292711 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f6a4f8-f8d7-4d7c-82de-a3d9a4affe2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:28:59.292988 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f6a4f8-f8d7-4d7c-82de-a3d9a4affe2a admin admin] PATH_INFO: `/v3/projects` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:28:59.293727 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f6a4f8-f8d7-4d7c-82de-a3d9a4affe2a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:28:59.295831 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f6a4f8-f8d7-4d7c-82de-a3d9a4affe2a admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:28:59.428037 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:28:59.443772 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1249/2472] 10.222.0.23 () {60 vars in 1152 bytes} [Thu May 19 12:28:59 2022] POST /identity/v3/projects => generated 371 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 12:28:59.452130 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337df783-6dba-495d-8d32-cf2dd44eec1e admin admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:28:59.462153 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337df783-6dba-495d-8d32-cf2dd44eec1e admin admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:28:59.565454 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcdca69-55c8-46d6-9f08-7434f9e00905 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:28:59.566225 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcdca69-55c8-46d6-9f08-7434f9e00905 admin admin] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:28:59.566555 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcdca69-55c8-46d6-9f08-7434f9e00905 admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:28:59.566880 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcdca69-55c8-46d6-9f08-7434f9e00905 admin admin] PATH_INFO: `/v3/users` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:28:59.567636 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bcdca69-55c8-46d6-9f08-7434f9e00905 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:28:59.569990 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bcdca69-55c8-46d6-9f08-7434f9e00905 admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:28:59.759038 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1224/2473] 10.222.0.23 () {60 vars in 1143 bytes} [Thu May 19 12:28:59 2022] POST /identity/v3/users => generated 286 bytes in 315 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 12:28:59.764821 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f6a4f8-f8d7-4d7c-82de-a3d9a4affe2a admin admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:28:59.819331 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f6a4f8-f8d7-4d7c-82de-a3d9a4affe2a admin admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:28:59.828238 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0ff6c2-c7b8-412d-a883-2e8964da14c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:28:59.829372 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0ff6c2-c7b8-412d-a883-2e8964da14c8 admin admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:28:59.829898 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0ff6c2-c7b8-412d-a883-2e8964da14c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:28:59.830401 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0ff6c2-c7b8-412d-a883-2e8964da14c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:28:59.831845 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0ff6c2-c7b8-412d-a883-2e8964da14c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:28:59.834856 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0ff6c2-c7b8-412d-a883-2e8964da14c8 admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:28:59.854830 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1250/2474] 10.222.0.23 () {58 vars in 1121 bytes} [Thu May 19 12:28:59 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 19 12:28:59.861793 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcdca69-55c8-46d6-9f08-7434f9e00905 admin admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:28:59.881132 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcdca69-55c8-46d6-9f08-7434f9e00905 admin admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:28:59.890464 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d392c5a6-92fb-4053-b1c4-48c3c3286d18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:28:59.898436 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d392c5a6-92fb-4053-b1c4-48c3c3286d18 admin admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:28:59.898436 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d392c5a6-92fb-4053-b1c4-48c3c3286d18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:28:59.898436 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d392c5a6-92fb-4053-b1c4-48c3c3286d18 admin admin] PATH_INFO: `/v3/roles` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:28:59.898436 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d392c5a6-92fb-4053-b1c4-48c3c3286d18 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:28:59.898436 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d392c5a6-92fb-4053-b1c4-48c3c3286d18 admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:28:59.919329 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1225/2475] 10.222.0.23 () {58 vars in 1121 bytes} [Thu May 19 12:28:59 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 12:28:59.923877 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0ff6c2-c7b8-412d-a883-2e8964da14c8 admin admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:28:59.935839 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0ff6c2-c7b8-412d-a883-2e8964da14c8 admin admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:28:59.941108 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09f6722-08a5-47be-97f5-2dacba30551e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:28:59.942022 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09f6722-08a5-47be-97f5-2dacba30551e admin admin] REQUEST_METHOD: `PUT` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:28:59.942306 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09f6722-08a5-47be-97f5-2dacba30551e admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:28:59.942580 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09f6722-08a5-47be-97f5-2dacba30551e admin admin] PATH_INFO: `/v3/projects/c9d4c8fb9a2949199059e7b6288325a3/users/b1b3f1fe24e144a392d1f036f9416315/roles/259eb400cdb4439db72d14456ae9cd08` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:28:59.964000 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09f6722-08a5-47be-97f5-2dacba30551e admin admin] RBAC: Authorizing `identity:create_grant(project_id=c9d4c8fb9a2949199059e7b6288325a3, user_id=b1b3f1fe24e144a392d1f036f9416315, role_id=259eb400cdb4439db72d14456ae9cd08)` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:28:59.967143 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09f6722-08a5-47be-97f5-2dacba30551e admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:29:00.215850 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1251/2476] 10.222.0.23 () {60 vars in 1482 bytes} [Thu May 19 12:28:59 2022] PUT /identity/v3/projects/c9d4c8fb9a2949199059e7b6288325a3/users/b1b3f1fe24e144a392d1f036f9416315/roles/259eb400cdb4439db72d14456ae9cd08 => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 12:29:00.229019 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94b65a2-041f-4756-a038-028da1a70fbf None None] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:29:00.230848 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94b65a2-041f-4756-a038-028da1a70fbf None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:29:00.230848 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94b65a2-041f-4756-a038-028da1a70fbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:29:00.264039 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.auth.core [None req-f94b65a2-041f-4756-a038-028da1a70fbf None None] MFA Rules not processed for user `b1b3f1fe24e144a392d1f036f9416315`. Rule list: `[]` (Enabled: `True`). {{(pid=80539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:29:00.267486 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-f94b65a2-041f-4756-a038-028da1a70fbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:29:00.367304 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1226/2477] 10.222.0.23 () {58 vars in 957 bytes} [Thu May 19 12:29:00 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:29:00.373293 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09f6722-08a5-47be-97f5-2dacba30551e admin admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:29:00.449814 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09f6722-08a5-47be-97f5-2dacba30551e admin admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:29:00.455051 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253a1fcc-4e00-40cf-b0e7-b1880196a8fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:29:00.455748 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253a1fcc-4e00-40cf-b0e7-b1880196a8fc admin admin] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:29:00.456093 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253a1fcc-4e00-40cf-b0e7-b1880196a8fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:29:00.456380 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253a1fcc-4e00-40cf-b0e7-b1880196a8fc admin admin] PATH_INFO: `/v3/projects` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:29:00.457092 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253a1fcc-4e00-40cf-b0e7-b1880196a8fc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:29:00.459064 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253a1fcc-4e00-40cf-b0e7-b1880196a8fc admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:29:00.681038 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1252/2478] 10.222.0.23 () {60 vars in 1152 bytes} [Thu May 19 12:29:00 2022] POST /identity/v3/projects => generated 371 bytes in 309 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 12:29:00.687704 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94b65a2-041f-4756-a038-028da1a70fbf None None] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:29:00.705993 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94b65a2-041f-4756-a038-028da1a70fbf None None] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:29:00.721945 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:29:00.799304 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8794553d-670b-4069-aa7d-0a60d9940664 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:29:00.800510 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8794553d-670b-4069-aa7d-0a60d9940664 admin admin] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:29:00.801015 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8794553d-670b-4069-aa7d-0a60d9940664 admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:29:00.801523 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8794553d-670b-4069-aa7d-0a60d9940664 admin admin] PATH_INFO: `/v3/users` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:29:00.802961 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8794553d-670b-4069-aa7d-0a60d9940664 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:29:00.806648 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8794553d-670b-4069-aa7d-0a60d9940664 admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:29:00.989500 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1227/2479] 10.222.0.23 () {60 vars in 1143 bytes} [Thu May 19 12:29:00 2022] POST /identity/v3/users => generated 292 bytes in 303 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 12:29:00.993984 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253a1fcc-4e00-40cf-b0e7-b1880196a8fc admin admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:29:01.006621 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253a1fcc-4e00-40cf-b0e7-b1880196a8fc admin admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:29:01.012433 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad428da-0670-4b31-9f8c-07ee0b27ddce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:29:01.013684 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad428da-0670-4b31-9f8c-07ee0b27ddce admin admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:29:01.013684 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad428da-0670-4b31-9f8c-07ee0b27ddce admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:29:01.013684 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad428da-0670-4b31-9f8c-07ee0b27ddce admin admin] PATH_INFO: `/v3/roles` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:29:01.014484 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad428da-0670-4b31-9f8c-07ee0b27ddce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:29:01.016256 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad428da-0670-4b31-9f8c-07ee0b27ddce admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:29:01.029053 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1253/2480] 10.222.0.23 () {58 vars in 1121 bytes} [Thu May 19 12:29:00 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 12:29:01.034158 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8794553d-670b-4069-aa7d-0a60d9940664 admin admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:29:01.047249 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8794553d-670b-4069-aa7d-0a60d9940664 admin admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:29:01.053441 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40661d6b-c8a6-4c68-a139-a7950bfccda5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:29:01.054416 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40661d6b-c8a6-4c68-a139-a7950bfccda5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:29:01.054781 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40661d6b-c8a6-4c68-a139-a7950bfccda5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:29:01.055118 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40661d6b-c8a6-4c68-a139-a7950bfccda5 admin admin] PATH_INFO: `/v3/projects/70656efb55db4a2a86d7c1b974127cde/users/dbf54268be5146e09971c8347e90de6b/roles/9db5671c65d3414ea2682afa2ccf1403` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:29:01.073939 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40661d6b-c8a6-4c68-a139-a7950bfccda5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=70656efb55db4a2a86d7c1b974127cde, user_id=dbf54268be5146e09971c8347e90de6b, role_id=9db5671c65d3414ea2682afa2ccf1403)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:29:01.076875 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40661d6b-c8a6-4c68-a139-a7950bfccda5 admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:29:01.201538 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1228/2481] 10.222.0.23 () {60 vars in 1482 bytes} [Thu May 19 12:29:01 2022] PUT /identity/v3/projects/70656efb55db4a2a86d7c1b974127cde/users/dbf54268be5146e09971c8347e90de6b/roles/9db5671c65d3414ea2682afa2ccf1403 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:29:01.206246 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad428da-0670-4b31-9f8c-07ee0b27ddce admin admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:29:01.219839 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad428da-0670-4b31-9f8c-07ee0b27ddce admin admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:29:01.312516 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd77841-79db-42e7-8a01-d9a4080c8fec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:29:01.313778 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd77841-79db-42e7-8a01-d9a4080c8fec admin admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:29:01.314222 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd77841-79db-42e7-8a01-d9a4080c8fec admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:29:01.314695 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd77841-79db-42e7-8a01-d9a4080c8fec admin admin] PATH_INFO: `/v3/roles` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:29:01.316081 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd77841-79db-42e7-8a01-d9a4080c8fec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:29:01.319416 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd77841-79db-42e7-8a01-d9a4080c8fec admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:29:01.340741 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1254/2482] 10.222.0.23 () {58 vars in 1121 bytes} [Thu May 19 12:29:01 2022] GET /identity/v3/roles => generated 1420 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 19 12:29:01.347080 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40661d6b-c8a6-4c68-a139-a7950bfccda5 admin admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:29:01.366431 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40661d6b-c8a6-4c68-a139-a7950bfccda5 admin admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:29:01.374128 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27b6e94-966e-4ba2-b986-deea46256e5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:29:01.376115 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27b6e94-966e-4ba2-b986-deea46256e5f admin admin] REQUEST_METHOD: `PUT` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:29:01.376733 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27b6e94-966e-4ba2-b986-deea46256e5f admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:29:01.377517 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27b6e94-966e-4ba2-b986-deea46256e5f admin admin] PATH_INFO: `/v3/projects/70656efb55db4a2a86d7c1b974127cde/users/dbf54268be5146e09971c8347e90de6b/roles/9db5671c65d3414ea2682afa2ccf1403` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:29:01.381503 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27b6e94-966e-4ba2-b986-deea46256e5f admin admin] RBAC: Authorizing `identity:create_grant(project_id=70656efb55db4a2a86d7c1b974127cde, user_id=dbf54268be5146e09971c8347e90de6b, role_id=9db5671c65d3414ea2682afa2ccf1403)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:29:01.387493 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27b6e94-966e-4ba2-b986-deea46256e5f admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:29:01.409458 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1229/2483] 10.222.0.23 () {60 vars in 1482 bytes} [Thu May 19 12:29:01 2022] PUT /identity/v3/projects/70656efb55db4a2a86d7c1b974127cde/users/dbf54268be5146e09971c8347e90de6b/roles/9db5671c65d3414ea2682afa2ccf1403 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 12:29:01.429057 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a89dbd-ed38-4b0e-a01e-a16c4da43a76 None None] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:29:01.429540 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a89dbd-ed38-4b0e-a01e-a16c4da43a76 None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:29:01.430041 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a89dbd-ed38-4b0e-a01e-a16c4da43a76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:29:01.452282 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:29:01.463717 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.auth.core [None req-17a89dbd-ed38-4b0e-a01e-a16c4da43a76 None None] MFA Rules not processed for user `dbf54268be5146e09971c8347e90de6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:29:01.465974 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-17a89dbd-ed38-4b0e-a01e-a16c4da43a76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:29:01.553521 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1255/2484] 10.222.0.23 () {58 vars in 957 bytes} [Thu May 19 12:29:01 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:29:01.571574 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27b6e94-966e-4ba2-b986-deea46256e5f admin admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:29:01.574308 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-c27b6e94-966e-4ba2-b986-deea46256e5f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:29:01.676922 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27b6e94-966e-4ba2-b986-deea46256e5f admin admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:29:01.685268 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596899d2-895d-460b-a98b-af059dc9e0c8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8800254c0e784e42a791fb9abc274d79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd70d25bf69314820b8aaecb65576978a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:29:01.685951 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596899d2-895d-460b-a98b-af059dc9e0c8 service cinder] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:29:01.686130 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596899d2-895d-460b-a98b-af059dc9e0c8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:29:01.686548 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596899d2-895d-460b-a98b-af059dc9e0c8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:29:01.702057 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596899d2-895d-460b-a98b-af059dc9e0c8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:29:01.703968 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596899d2-895d-460b-a98b-af059dc9e0c8 service cinder] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:29:01.719162 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1230/2485] 10.222.0.23 () {60 vars in 1365 bytes} [Thu May 19 12:29:01 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:29:01.786526 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792307af-65c4-4152-83d7-14ab22e17982 None None] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:29:01.786978 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792307af-65c4-4152-83d7-14ab22e17982 None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:29:01.787358 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792307af-65c4-4152-83d7-14ab22e17982 None None] PATH_INFO: `/` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:29:01.789001 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1256/2486] 10.222.0.23 () {54 vars in 921 bytes} [Thu May 19 12:29:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 12:29:01.796582 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4898d9-a3ea-49f8-9a96-0c0e92a4274e None None] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:29:01.796890 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4898d9-a3ea-49f8-9a96-0c0e92a4274e None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:29:01.797174 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4898d9-a3ea-49f8-9a96-0c0e92a4274e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:29:01.817640 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.auth.core [None req-bb4898d9-a3ea-49f8-9a96-0c0e92a4274e None None] MFA Rules not processed for user `34a98d75d84b4c67981377e0f26eca62`. Rule list: `[]` (Enabled: `True`). {{(pid=80539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:29:01.818955 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-bb4898d9-a3ea-49f8-9a96-0c0e92a4274e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:29:01.880106 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:29:01.918252 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1231/2487] 10.222.0.23 () {58 vars in 1018 bytes} [Thu May 19 12:29:01 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:29:01.934859 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792307af-65c4-4152-83d7-14ab22e17982 None None] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:29:01.941525 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-792307af-65c4-4152-83d7-14ab22e17982 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:29:02.010270 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792307af-65c4-4152-83d7-14ab22e17982 None None] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:29:02.014066 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fdfe1e-32c0-45d0-83ff-37afbfdfb7d9 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '21697802f2be42f9baa8e07490d5ebdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd70d25bf69314820b8aaecb65576978a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:29:02.014747 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fdfe1e-32c0-45d0-83ff-37afbfdfb7d9 service glance] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:29:02.015040 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fdfe1e-32c0-45d0-83ff-37afbfdfb7d9 service glance] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:29:02.015379 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fdfe1e-32c0-45d0-83ff-37afbfdfb7d9 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:29:02.028634 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81fdfe1e-32c0-45d0-83ff-37afbfdfb7d9 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:29:02.030727 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81fdfe1e-32c0-45d0-83ff-37afbfdfb7d9 service glance] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:29:02.043843 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1257/2488] 10.222.0.23 () {60 vars in 1365 bytes} [Thu May 19 12:29:01 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:29:02.090297 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4898d9-a3ea-49f8-9a96-0c0e92a4274e None None] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:29:02.101855 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4898d9-a3ea-49f8-9a96-0c0e92a4274e None None] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:29:02.106944 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1903cbe6-5776-405b-bdce-1573ddbbb6e2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8800254c0e784e42a791fb9abc274d79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd70d25bf69314820b8aaecb65576978a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:29:02.107734 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1903cbe6-5776-405b-bdce-1573ddbbb6e2 service cinder] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:29:02.108157 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1903cbe6-5776-405b-bdce-1573ddbbb6e2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:29:02.108548 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1903cbe6-5776-405b-bdce-1573ddbbb6e2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:29:02.172902 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1903cbe6-5776-405b-bdce-1573ddbbb6e2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:29:02.174401 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1903cbe6-5776-405b-bdce-1573ddbbb6e2 service cinder] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:29:02.185837 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1232/2489] 10.222.0.23 () {60 vars in 1365 bytes} [Thu May 19 12:29:02 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:29:02.389873 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a7a5ab-76df-4624-872e-7d5cadfdb738 None None] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:29:02.390144 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a7a5ab-76df-4624-872e-7d5cadfdb738 None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:29:02.390430 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a7a5ab-76df-4624-872e-7d5cadfdb738 None None] PATH_INFO: `/` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:29:02.391951 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1258/2490] 10.222.0.23 () {54 vars in 921 bytes} [Thu May 19 12:29:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 12:29:02.398017 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3adb04e8-d639-4bef-bd68-d989b4d8ec68 None None] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:29:02.398902 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3adb04e8-d639-4bef-bd68-d989b4d8ec68 None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:29:02.398902 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3adb04e8-d639-4bef-bd68-d989b4d8ec68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:29:02.445203 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.auth.core [None req-3adb04e8-d639-4bef-bd68-d989b4d8ec68 None None] MFA Rules not processed for user `1a00b3fa7497456895e5e0bf105c4fc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:29:02.446514 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-3adb04e8-d639-4bef-bd68-d989b4d8ec68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:29:02.525168 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1233/2491] 10.222.0.23 () {58 vars in 1018 bytes} [Thu May 19 12:29:02 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:29:02.540580 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a7a5ab-76df-4624-872e-7d5cadfdb738 None None] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:29:02.543256 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-09a7a5ab-76df-4624-872e-7d5cadfdb738 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:29:02.561838 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a7a5ab-76df-4624-872e-7d5cadfdb738 None None] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:29:02.567468 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d860373a-0a0a-429e-a81c-40e6a5d17c84 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a00b3fa7497456895e5e0bf105c4fc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd70d25bf69314820b8aaecb65576978a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:29:02.568388 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d860373a-0a0a-429e-a81c-40e6a5d17c84 service neutron] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:29:02.568832 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d860373a-0a0a-429e-a81c-40e6a5d17c84 service neutron] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:29:02.569230 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d860373a-0a0a-429e-a81c-40e6a5d17c84 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:29:02.579665 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d860373a-0a0a-429e-a81c-40e6a5d17c84 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:29:02.581206 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d860373a-0a0a-429e-a81c-40e6a5d17c84 service neutron] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:29:02.591738 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1259/2492] 10.222.0.23 () {60 vars in 1365 bytes} [Thu May 19 12:29:02 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:29:02.697854 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec18858-21e4-408b-bab8-f10fef9ccf82 None None] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:29:02.698367 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec18858-21e4-408b-bab8-f10fef9ccf82 None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:29:02.698899 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec18858-21e4-408b-bab8-f10fef9ccf82 None None] PATH_INFO: `/` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:29:02.700862 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1234/2493] 10.222.0.23 () {54 vars in 921 bytes} [Thu May 19 12:29:02 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 12:29:02.708799 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff806f93-0a4e-4221-97eb-f44fc29e33fb None None] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:29:02.709181 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff806f93-0a4e-4221-97eb-f44fc29e33fb None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:29:02.709540 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff806f93-0a4e-4221-97eb-f44fc29e33fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:29:02.734466 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.auth.core [None req-ff806f93-0a4e-4221-97eb-f44fc29e33fb None None] MFA Rules not processed for user `34a98d75d84b4c67981377e0f26eca62`. Rule list: `[]` (Enabled: `True`). {{(pid=80540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:29:02.736058 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-ff806f93-0a4e-4221-97eb-f44fc29e33fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:29:02.748743 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1260/2494] 10.222.0.23 () {58 vars in 1018 bytes} [Thu May 19 12:29:02 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:29:02.771836 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec18858-21e4-408b-bab8-f10fef9ccf82 None None] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:29:02.775081 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-5ec18858-21e4-408b-bab8-f10fef9ccf82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:29:02.801784 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec18858-21e4-408b-bab8-f10fef9ccf82 None None] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:29:02.808786 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc8a2d3-c399-4fff-918a-f5c39862c9da service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a00b3fa7497456895e5e0bf105c4fc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd70d25bf69314820b8aaecb65576978a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:29:02.810231 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc8a2d3-c399-4fff-918a-f5c39862c9da service neutron] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:29:02.810725 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc8a2d3-c399-4fff-918a-f5c39862c9da service neutron] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:29:02.811218 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc8a2d3-c399-4fff-918a-f5c39862c9da service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:29:02.828843 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc8a2d3-c399-4fff-918a-f5c39862c9da service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:29:02.830333 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc8a2d3-c399-4fff-918a-f5c39862c9da service neutron] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:29:02.844046 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1235/2495] 10.222.0.23 () {60 vars in 1365 bytes} [Thu May 19 12:29:02 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 74 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 12:29:02.944443 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff806f93-0a4e-4221-97eb-f44fc29e33fb None None] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:29:02.946742 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-ff806f93-0a4e-4221-97eb-f44fc29e33fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:29:03.044905 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff806f93-0a4e-4221-97eb-f44fc29e33fb None None] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:29:03.050527 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e24ab9e-8c5b-46bb-a1a1-dba6d486669e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50f937eed4f3414fa6f19cab2a84eb5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd70d25bf69314820b8aaecb65576978a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:29:03.051207 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e24ab9e-8c5b-46bb-a1a1-dba6d486669e service placement] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:29:03.051494 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e24ab9e-8c5b-46bb-a1a1-dba6d486669e service placement] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:29:03.051774 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e24ab9e-8c5b-46bb-a1a1-dba6d486669e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:29:03.053898 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-2e24ab9e-8c5b-46bb-a1a1-dba6d486669e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:29:03.067766 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e24ab9e-8c5b-46bb-a1a1-dba6d486669e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:29:03.069640 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e24ab9e-8c5b-46bb-a1a1-dba6d486669e service placement] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:29:03.082978 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1261/2496] 10.222.0.23 () {60 vars in 1365 bytes} [Thu May 19 12:29:02 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:29:06.040304 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bade5719-c257-4cf0-96d5-fe3a653f1327 None None] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:29:06.041146 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bade5719-c257-4cf0-96d5-fe3a653f1327 None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:29:06.041146 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bade5719-c257-4cf0-96d5-fe3a653f1327 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:29:06.063099 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.auth.core [None req-bade5719-c257-4cf0-96d5-fe3a653f1327 None None] MFA Rules not processed for user `0c3181fb4f584a549a786adfefdf3dfb`. Rule list: `[]` (Enabled: `True`). {{(pid=80539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:29:06.064815 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-bade5719-c257-4cf0-96d5-fe3a653f1327 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:29:06.132627 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1236/2497] 10.222.0.23 () {58 vars in 1009 bytes} [Thu May 19 12:29:06 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 12:29:06.146947 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e24ab9e-8c5b-46bb-a1a1-dba6d486669e service placement] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:29:06.148848 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-2e24ab9e-8c5b-46bb-a1a1-dba6d486669e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:29:06.161656 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:29:06.240763 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e24ab9e-8c5b-46bb-a1a1-dba6d486669e service placement] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:29:06.247272 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20865d49-1d92-40a0-9ede-0c903283299c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15cbfc73ae1243d2a8b1e70f9cd99876', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd70d25bf69314820b8aaecb65576978a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:29:06.247985 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20865d49-1d92-40a0-9ede-0c903283299c service swift] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:29:06.248275 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20865d49-1d92-40a0-9ede-0c903283299c service swift] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:29:06.248556 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20865d49-1d92-40a0-9ede-0c903283299c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:29:06.258427 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20865d49-1d92-40a0-9ede-0c903283299c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:29:06.260212 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20865d49-1d92-40a0-9ede-0c903283299c service swift] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:29:06.271355 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1262/2498] 10.222.0.23 () {60 vars in 1384 bytes} [Thu May 19 12:29:06 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) May 19 12:29:16.124434 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=98 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:29:16.267042 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=29 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:29:27.530640 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bade5719-c257-4cf0-96d5-fe3a653f1327 None None] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:29:27.533122 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-bade5719-c257-4cf0-96d5-fe3a653f1327 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:29:27.553306 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bade5719-c257-4cf0-96d5-fe3a653f1327 None None] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:29:27.559564 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9cbcce6-5732-44be-a6dd-158c180fdcdd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34a98d75d84b4c67981377e0f26eca62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd70d25bf69314820b8aaecb65576978a', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:29:27.560434 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cbcce6-5732-44be-a6dd-158c180fdcdd service nova] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:29:27.560769 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cbcce6-5732-44be-a6dd-158c180fdcdd service nova] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:29:27.561112 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cbcce6-5732-44be-a6dd-158c180fdcdd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:29:27.563662 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-a9cbcce6-5732-44be-a6dd-158c180fdcdd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:29:27.584577 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9cbcce6-5732-44be-a6dd-158c180fdcdd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:29:27.587026 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9cbcce6-5732-44be-a6dd-158c180fdcdd service nova] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:29:27.601679 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1237/2499] 10.222.0.23 () {60 vars in 1365 bytes} [Thu May 19 12:29:27 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 72 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 12:29:29.770042 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512b764a-9035-486f-9d3b-afcc04e72773 None None] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:29:29.770635 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512b764a-9035-486f-9d3b-afcc04e72773 None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:29:29.770635 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512b764a-9035-486f-9d3b-afcc04e72773 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:29:29.791832 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.auth.core [None req-512b764a-9035-486f-9d3b-afcc04e72773 None None] MFA Rules not processed for user `0c3181fb4f584a549a786adfefdf3dfb`. Rule list: `[]` (Enabled: `True`). {{(pid=80540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:29:29.797165 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-512b764a-9035-486f-9d3b-afcc04e72773 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:29:29.804118 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1263/2500] 10.222.0.23 () {58 vars in 967 bytes} [Thu May 19 12:29:29 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:29:29.818208 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9cbcce6-5732-44be-a6dd-158c180fdcdd service nova] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:29:29.830858 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9cbcce6-5732-44be-a6dd-158c180fdcdd service nova] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:29:29.836202 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3f26f4-84a2-4834-9753-730218034387 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15cbfc73ae1243d2a8b1e70f9cd99876', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd70d25bf69314820b8aaecb65576978a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:29:29.836887 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3f26f4-84a2-4834-9753-730218034387 service swift] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:29:29.837170 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3f26f4-84a2-4834-9753-730218034387 service swift] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:29:29.837450 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3f26f4-84a2-4834-9753-730218034387 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:29:29.849660 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a3f26f4-84a2-4834-9753-730218034387 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:29:29.851299 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a3f26f4-84a2-4834-9753-730218034387 service swift] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:29:29.862080 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1238/2501] 10.222.0.23 () {60 vars in 1384 bytes} [Thu May 19 12:29:29 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) May 19 12:29:39.785287 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=3 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:29:39.857904 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=12 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:30:02.488272 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512b764a-9035-486f-9d3b-afcc04e72773 None None] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:30:02.501913 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512b764a-9035-486f-9d3b-afcc04e72773 None None] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:30:02.507662 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f9ad40-f5ce-4f64-9f04-0b821fcc5d76 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34a98d75d84b4c67981377e0f26eca62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd70d25bf69314820b8aaecb65576978a', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:30:02.508360 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f9ad40-f5ce-4f64-9f04-0b821fcc5d76 service nova] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:30:02.508645 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f9ad40-f5ce-4f64-9f04-0b821fcc5d76 service nova] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:30:02.508921 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f9ad40-f5ce-4f64-9f04-0b821fcc5d76 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:30:02.511486 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-79f9ad40-f5ce-4f64-9f04-0b821fcc5d76 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:30:02.605562 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f9ad40-f5ce-4f64-9f04-0b821fcc5d76 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:30:02.607705 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f9ad40-f5ce-4f64-9f04-0b821fcc5d76 service nova] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:30:02.628701 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1264/2502] 10.222.0.23 () {60 vars in 1365 bytes} [Thu May 19 12:30:02 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 12:30:10.010756 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3f26f4-84a2-4834-9753-730218034387 service swift] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:30:10.102758 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3f26f4-84a2-4834-9753-730218034387 service swift] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:30:10.109086 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee02c11-8290-449f-8781-de97f7eea6fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:30:10.109982 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee02c11-8290-449f-8781-de97f7eea6fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:30:10.110266 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee02c11-8290-449f-8781-de97f7eea6fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:30:10.110539 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee02c11-8290-449f-8781-de97f7eea6fe admin admin] PATH_INFO: `/v3/users/b1b3f1fe24e144a392d1f036f9416315` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:30:10.112084 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee02c11-8290-449f-8781-de97f7eea6fe admin admin] RBAC: Authorizing `identity:delete_user(user_id=b1b3f1fe24e144a392d1f036f9416315)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:30:10.114024 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee02c11-8290-449f-8781-de97f7eea6fe admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:30:10.351852 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-9ee02c11-8290-449f-8781-de97f7eea6fe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b1b3f1fe24e144a392d1f036f9416315'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:30:10.366069 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-9ee02c11-8290-449f-8781-de97f7eea6fe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b1b3f1fe24e144a392d1f036f9416315'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:30:10.366756 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-9ee02c11-8290-449f-8781-de97f7eea6fe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b1b3f1fe24e144a392d1f036f9416315'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:30:10.471522 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-9ee02c11-8290-449f-8781-de97f7eea6fe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b1b3f1fe24e144a392d1f036f9416315'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:30:10.481211 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-9ee02c11-8290-449f-8781-de97f7eea6fe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b1b3f1fe24e144a392d1f036f9416315'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:30:10.488413 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1239/2503] 10.222.0.23 () {58 vars in 1223 bytes} [Thu May 19 12:30:10 2022] DELETE /identity/v3/users/b1b3f1fe24e144a392d1f036f9416315 => generated 0 bytes in 478 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:30:10.493296 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438c827b-7bb7-45b1-ac30-95d77303a71f None None] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:30:10.493951 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438c827b-7bb7-45b1-ac30-95d77303a71f None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:30:10.493951 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438c827b-7bb7-45b1-ac30-95d77303a71f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:30:10.556229 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.auth.core [None req-438c827b-7bb7-45b1-ac30-95d77303a71f None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:30:10.558626 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-438c827b-7bb7-45b1-ac30-95d77303a71f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:30:10.656844 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1265/2504] 10.222.0.23 () {58 vars in 957 bytes} [Thu May 19 12:30:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:30:10.666817 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee02c11-8290-449f-8781-de97f7eea6fe admin admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:30:10.668618 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-9ee02c11-8290-449f-8781-de97f7eea6fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:30:10.744475 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:30:10.754512 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee02c11-8290-449f-8781-de97f7eea6fe admin admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:30:10.758333 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eba22f3-e687-474f-abe4-f9c3185208a3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a00b3fa7497456895e5e0bf105c4fc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd70d25bf69314820b8aaecb65576978a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:30:10.759001 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eba22f3-e687-474f-abe4-f9c3185208a3 service neutron] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:30:10.759265 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eba22f3-e687-474f-abe4-f9c3185208a3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:30:10.759538 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eba22f3-e687-474f-abe4-f9c3185208a3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:30:10.770253 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eba22f3-e687-474f-abe4-f9c3185208a3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:30:10.771695 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eba22f3-e687-474f-abe4-f9c3185208a3 service neutron] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:30:10.784248 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1240/2505] 10.222.0.23 () {60 vars in 1365 bytes} [Thu May 19 12:30:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:30:11.315009 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438c827b-7bb7-45b1-ac30-95d77303a71f None None] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:30:11.319215 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-438c827b-7bb7-45b1-ac30-95d77303a71f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:30:11.347211 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438c827b-7bb7-45b1-ac30-95d77303a71f None None] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:30:11.360023 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eaada03-8573-4f98-a689-6853edbc5acf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:30:11.360023 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eaada03-8573-4f98-a689-6853edbc5acf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:30:11.360023 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eaada03-8573-4f98-a689-6853edbc5acf admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:30:11.360023 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eaada03-8573-4f98-a689-6853edbc5acf admin admin] PATH_INFO: `/v3/projects/c9d4c8fb9a2949199059e7b6288325a3` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:30:11.360558 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eaada03-8573-4f98-a689-6853edbc5acf admin admin] RBAC: Authorizing `identity:delete_project(project_id=c9d4c8fb9a2949199059e7b6288325a3)` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:30:11.368922 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eaada03-8573-4f98-a689-6853edbc5acf admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:30:11.527001 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:30:11.527001 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-3eaada03-8573-4f98-a689-6853edbc5acf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c9d4c8fb9a2949199059e7b6288325a3', 'request_id': 'req-3eaada03-8573-4f98-a689-6853edbc5acf'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:30:11.532139 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-3eaada03-8573-4f98-a689-6853edbc5acf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c9d4c8fb9a2949199059e7b6288325a3', 'request_id': 'req-3eaada03-8573-4f98-a689-6853edbc5acf'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:30:11.540357 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-3eaada03-8573-4f98-a689-6853edbc5acf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c9d4c8fb9a2949199059e7b6288325a3', 'request_id': 'req-3eaada03-8573-4f98-a689-6853edbc5acf'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:30:11.665332 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-3eaada03-8573-4f98-a689-6853edbc5acf admin admin] The token cache is being invalidate because project c9d4c8fb9a2949199059e7b6288325a3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80540) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 12:30:11.665617 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-3eaada03-8573-4f98-a689-6853edbc5acf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:30:11.666269 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-3eaada03-8573-4f98-a689-6853edbc5acf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:30:11.667930 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1266/2506] 10.222.0.23 () {58 vars in 1232 bytes} [Thu May 19 12:30:11 2022] DELETE /identity/v3/projects/c9d4c8fb9a2949199059e7b6288325a3 => generated 0 bytes in 354 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:30:11.672361 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eba22f3-e687-474f-abe4-f9c3185208a3 service neutron] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:30:11.686362 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eba22f3-e687-474f-abe4-f9c3185208a3 service neutron] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:30:11.749223 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578a0944-1dcd-463a-bfb5-61b18f7c9543 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:30:11.750061 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578a0944-1dcd-463a-bfb5-61b18f7c9543 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:30:11.750347 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578a0944-1dcd-463a-bfb5-61b18f7c9543 admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:30:11.750641 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578a0944-1dcd-463a-bfb5-61b18f7c9543 admin admin] PATH_INFO: `/v3/users/dbf54268be5146e09971c8347e90de6b` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:30:11.752100 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578a0944-1dcd-463a-bfb5-61b18f7c9543 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dbf54268be5146e09971c8347e90de6b)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:30:11.754065 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578a0944-1dcd-463a-bfb5-61b18f7c9543 admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:30:12.018403 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-578a0944-1dcd-463a-bfb5-61b18f7c9543 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dbf54268be5146e09971c8347e90de6b'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:30:12.035096 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-578a0944-1dcd-463a-bfb5-61b18f7c9543 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dbf54268be5146e09971c8347e90de6b'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:30:12.035762 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-578a0944-1dcd-463a-bfb5-61b18f7c9543 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dbf54268be5146e09971c8347e90de6b'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:30:12.180397 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-578a0944-1dcd-463a-bfb5-61b18f7c9543 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dbf54268be5146e09971c8347e90de6b'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:30:12.190546 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-578a0944-1dcd-463a-bfb5-61b18f7c9543 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dbf54268be5146e09971c8347e90de6b'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:30:12.197401 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1241/2507] 10.222.0.23 () {58 vars in 1223 bytes} [Thu May 19 12:30:11 2022] DELETE /identity/v3/users/dbf54268be5146e09971c8347e90de6b => generated 0 bytes in 526 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:30:12.772506 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eaada03-8573-4f98-a689-6853edbc5acf admin admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:30:12.774494 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-3eaada03-8573-4f98-a689-6853edbc5acf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:30:12.791475 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eaada03-8573-4f98-a689-6853edbc5acf admin admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:30:12.870760 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d944219d-699d-489e-b279-5ce830da3bc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:30:12.871662 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d944219d-699d-489e-b279-5ce830da3bc3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:30:12.872007 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d944219d-699d-489e-b279-5ce830da3bc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:30:12.872295 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d944219d-699d-489e-b279-5ce830da3bc3 admin admin] PATH_INFO: `/v3/projects/70656efb55db4a2a86d7c1b974127cde` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:30:12.873595 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d944219d-699d-489e-b279-5ce830da3bc3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=70656efb55db4a2a86d7c1b974127cde)` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:30:12.875562 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d944219d-699d-489e-b279-5ce830da3bc3 admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:30:12.996853 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-d944219d-699d-489e-b279-5ce830da3bc3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '70656efb55db4a2a86d7c1b974127cde', 'request_id': 'req-d944219d-699d-489e-b279-5ce830da3bc3'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:30:13.004611 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-d944219d-699d-489e-b279-5ce830da3bc3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '70656efb55db4a2a86d7c1b974127cde', 'request_id': 'req-d944219d-699d-489e-b279-5ce830da3bc3'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:30:13.011638 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-d944219d-699d-489e-b279-5ce830da3bc3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '70656efb55db4a2a86d7c1b974127cde', 'request_id': 'req-d944219d-699d-489e-b279-5ce830da3bc3'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:30:13.079221 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-d944219d-699d-489e-b279-5ce830da3bc3 admin admin] The token cache is being invalidate because project 70656efb55db4a2a86d7c1b974127cde was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80540) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 12:30:13.079515 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-d944219d-699d-489e-b279-5ce830da3bc3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:30:13.080197 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-d944219d-699d-489e-b279-5ce830da3bc3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:30:13.081840 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1267/2508] 10.222.0.23 () {58 vars in 1232 bytes} [Thu May 19 12:30:12 2022] DELETE /identity/v3/projects/70656efb55db4a2a86d7c1b974127cde => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:30:13.087081 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b5e90c-bfdb-411c-83d6-645b8e105c92 None None] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:30:13.087601 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b5e90c-bfdb-411c-83d6-645b8e105c92 None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:30:13.087701 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b5e90c-bfdb-411c-83d6-645b8e105c92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:30:13.163182 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.auth.core [None req-48b5e90c-bfdb-411c-83d6-645b8e105c92 None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:30:13.165381 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-48b5e90c-bfdb-411c-83d6-645b8e105c92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:30:13.227841 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:30:13.258965 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1242/2509] 10.222.0.23 () {58 vars in 957 bytes} [Thu May 19 12:30:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:30:13.267842 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedfc6f5-be47-4fbd-9b22-beb7d4965249 None None] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:30:13.268375 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedfc6f5-be47-4fbd-9b22-beb7d4965249 None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:30:13.268663 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedfc6f5-be47-4fbd-9b22-beb7d4965249 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:30:13.295325 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.auth.core [None req-cedfc6f5-be47-4fbd-9b22-beb7d4965249 None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:30:13.296610 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-cedfc6f5-be47-4fbd-9b22-beb7d4965249 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:30:13.305189 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1268/2510] 10.222.0.23 () {58 vars in 957 bytes} [Thu May 19 12:30:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:30:13.310390 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b5e90c-bfdb-411c-83d6-645b8e105c92 None None] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:30:13.320826 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b5e90c-bfdb-411c-83d6-645b8e105c92 None None] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:30:13.327650 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac33b9e-fbff-4d00-88cd-7a3cd98a7ab0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:30:13.328480 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac33b9e-fbff-4d00-88cd-7a3cd98a7ab0 admin admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:30:13.328834 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac33b9e-fbff-4d00-88cd-7a3cd98a7ab0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:30:13.329162 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac33b9e-fbff-4d00-88cd-7a3cd98a7ab0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:30:13.330003 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac33b9e-fbff-4d00-88cd-7a3cd98a7ab0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:30:13.330362 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac33b9e-fbff-4d00-88cd-7a3cd98a7ab0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:30:13.332137 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac33b9e-fbff-4d00-88cd-7a3cd98a7ab0 admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:30:13.347575 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1243/2511] 10.222.0.23 () {58 vars in 1152 bytes} [Thu May 19 12:30:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:30:13.352468 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cedfc6f5-be47-4fbd-9b22-beb7d4965249 None None] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:30:13.364541 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cedfc6f5-be47-4fbd-9b22-beb7d4965249 None None] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:30:13.371594 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9454e1-ff00-4016-82e4-8342a447c004 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:30:13.372447 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9454e1-ff00-4016-82e4-8342a447c004 admin admin] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:30:13.372840 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9454e1-ff00-4016-82e4-8342a447c004 admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:30:13.373222 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9454e1-ff00-4016-82e4-8342a447c004 admin admin] PATH_INFO: `/v3/projects` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:30:13.374042 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9454e1-ff00-4016-82e4-8342a447c004 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:30:13.377743 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9454e1-ff00-4016-82e4-8342a447c004 admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:30:13.469522 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1269/2512] 10.222.0.23 () {60 vars in 1152 bytes} [Thu May 19 12:30:13 2022] POST /identity/v3/projects => generated 367 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 12:30:13.474252 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac33b9e-fbff-4d00-88cd-7a3cd98a7ab0 admin admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:30:13.487771 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac33b9e-fbff-4d00-88cd-7a3cd98a7ab0 admin admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:30:13.558636 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09cfa03-70f0-43d7-93f1-a7bca23d73ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:30:13.559336 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09cfa03-70f0-43d7-93f1-a7bca23d73ad admin admin] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:30:13.559621 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09cfa03-70f0-43d7-93f1-a7bca23d73ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:30:13.559936 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09cfa03-70f0-43d7-93f1-a7bca23d73ad admin admin] PATH_INFO: `/v3/users` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:30:13.560658 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09cfa03-70f0-43d7-93f1-a7bca23d73ad admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:30:13.562625 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09cfa03-70f0-43d7-93f1-a7bca23d73ad admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:30:13.673043 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1244/2513] 10.222.0.23 () {60 vars in 1143 bytes} [Thu May 19 12:30:13 2022] POST /identity/v3/users => generated 284 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 12:30:13.679714 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9454e1-ff00-4016-82e4-8342a447c004 admin admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:30:13.699679 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9454e1-ff00-4016-82e4-8342a447c004 admin admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:30:13.707985 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3834bedb-c60d-4085-bfd2-7c99fcb72085 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:30:13.709130 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3834bedb-c60d-4085-bfd2-7c99fcb72085 admin admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:30:13.709652 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3834bedb-c60d-4085-bfd2-7c99fcb72085 admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:30:13.710207 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3834bedb-c60d-4085-bfd2-7c99fcb72085 admin admin] PATH_INFO: `/v3/roles` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:30:13.711618 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3834bedb-c60d-4085-bfd2-7c99fcb72085 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:30:13.715632 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3834bedb-c60d-4085-bfd2-7c99fcb72085 admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:30:13.735837 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1270/2514] 10.222.0.23 () {58 vars in 1121 bytes} [Thu May 19 12:30:13 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 12:30:13.742257 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09cfa03-70f0-43d7-93f1-a7bca23d73ad admin admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:30:13.756270 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09cfa03-70f0-43d7-93f1-a7bca23d73ad admin admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:30:13.761898 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807c921b-748e-4d61-84ca-3d01551f1cb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:30:13.762583 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807c921b-748e-4d61-84ca-3d01551f1cb0 admin admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:30:13.762867 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807c921b-748e-4d61-84ca-3d01551f1cb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:30:13.763141 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807c921b-748e-4d61-84ca-3d01551f1cb0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:30:13.763918 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807c921b-748e-4d61-84ca-3d01551f1cb0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:30:13.765550 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807c921b-748e-4d61-84ca-3d01551f1cb0 admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:30:13.777348 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1245/2515] 10.222.0.23 () {58 vars in 1121 bytes} [Thu May 19 12:30:13 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 12:30:13.782088 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3834bedb-c60d-4085-bfd2-7c99fcb72085 admin admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:30:13.793855 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3834bedb-c60d-4085-bfd2-7c99fcb72085 admin admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:30:13.798710 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168662c5-0669-4b9f-84cf-5ae3306cb5da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:30:13.799668 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168662c5-0669-4b9f-84cf-5ae3306cb5da admin admin] REQUEST_METHOD: `PUT` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:30:13.799978 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168662c5-0669-4b9f-84cf-5ae3306cb5da admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:30:13.800256 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168662c5-0669-4b9f-84cf-5ae3306cb5da admin admin] PATH_INFO: `/v3/projects/e03eae01937440af9454620be039539a/users/efce603b4efe4560b7509d32b92aa889/roles/259eb400cdb4439db72d14456ae9cd08` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:30:13.817599 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168662c5-0669-4b9f-84cf-5ae3306cb5da admin admin] RBAC: Authorizing `identity:create_grant(project_id=e03eae01937440af9454620be039539a, user_id=efce603b4efe4560b7509d32b92aa889, role_id=259eb400cdb4439db72d14456ae9cd08)` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:30:13.820436 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168662c5-0669-4b9f-84cf-5ae3306cb5da admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:30:13.956040 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1271/2516] 10.222.0.23 () {60 vars in 1482 bytes} [Thu May 19 12:30:13 2022] PUT /identity/v3/projects/e03eae01937440af9454620be039539a/users/efce603b4efe4560b7509d32b92aa889/roles/259eb400cdb4439db72d14456ae9cd08 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:30:13.969175 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838433ab-0249-48e0-a15e-20d2e5147c22 None None] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:30:13.969465 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838433ab-0249-48e0-a15e-20d2e5147c22 None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:30:13.969745 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838433ab-0249-48e0-a15e-20d2e5147c22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:30:14.008815 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.auth.core [None req-838433ab-0249-48e0-a15e-20d2e5147c22 None None] MFA Rules not processed for user `efce603b4efe4560b7509d32b92aa889`. Rule list: `[]` (Enabled: `True`). {{(pid=80539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:30:14.010072 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-838433ab-0249-48e0-a15e-20d2e5147c22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:30:14.115054 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1246/2517] 10.222.0.23 () {58 vars in 957 bytes} [Thu May 19 12:30:13 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 12:30:14.121449 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168662c5-0669-4b9f-84cf-5ae3306cb5da admin admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:30:14.126392 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:30:14.209488 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168662c5-0669-4b9f-84cf-5ae3306cb5da admin admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:30:14.217704 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7a4d5b-b43d-488d-b8fd-3aad5130560b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:30:14.218420 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7a4d5b-b43d-488d-b8fd-3aad5130560b admin admin] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:30:14.218847 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7a4d5b-b43d-488d-b8fd-3aad5130560b admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:30:14.219226 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7a4d5b-b43d-488d-b8fd-3aad5130560b admin admin] PATH_INFO: `/v3/projects` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:30:14.220106 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7a4d5b-b43d-488d-b8fd-3aad5130560b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:30:14.222801 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7a4d5b-b43d-488d-b8fd-3aad5130560b admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:30:14.373300 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1272/2518] 10.222.0.23 () {60 vars in 1152 bytes} [Thu May 19 12:30:14 2022] POST /identity/v3/projects => generated 369 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 12:30:14.378285 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838433ab-0249-48e0-a15e-20d2e5147c22 None None] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:30:14.390821 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838433ab-0249-48e0-a15e-20d2e5147c22 None None] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:30:14.431599 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:30:14.473226 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac0c1cd-bed9-4253-b972-1b102c15d8c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:30:14.480038 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac0c1cd-bed9-4253-b972-1b102c15d8c0 admin admin] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:30:14.480038 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac0c1cd-bed9-4253-b972-1b102c15d8c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:30:14.480038 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac0c1cd-bed9-4253-b972-1b102c15d8c0 admin admin] PATH_INFO: `/v3/users` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:30:14.480038 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac0c1cd-bed9-4253-b972-1b102c15d8c0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:30:14.480038 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac0c1cd-bed9-4253-b972-1b102c15d8c0 admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:30:14.642366 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1247/2519] 10.222.0.23 () {60 vars in 1143 bytes} [Thu May 19 12:30:14 2022] POST /identity/v3/users => generated 291 bytes in 265 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 12:30:14.647681 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7a4d5b-b43d-488d-b8fd-3aad5130560b admin admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:30:14.665523 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7a4d5b-b43d-488d-b8fd-3aad5130560b admin admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:30:14.674037 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362dbc09-704a-4bc7-b769-956d522e5a33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:30:14.675427 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362dbc09-704a-4bc7-b769-956d522e5a33 admin admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:30:14.676111 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362dbc09-704a-4bc7-b769-956d522e5a33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:30:14.676753 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362dbc09-704a-4bc7-b769-956d522e5a33 admin admin] PATH_INFO: `/v3/roles` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:30:14.678780 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-362dbc09-704a-4bc7-b769-956d522e5a33 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:30:14.683727 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-362dbc09-704a-4bc7-b769-956d522e5a33 admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:30:14.706270 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1273/2520] 10.222.0.23 () {58 vars in 1121 bytes} [Thu May 19 12:30:14 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 12:30:14.710518 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac0c1cd-bed9-4253-b972-1b102c15d8c0 admin admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:30:14.722327 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac0c1cd-bed9-4253-b972-1b102c15d8c0 admin admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:30:14.727350 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aede9e6a-6f42-4bad-b908-8057e81a5a2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:30:14.728503 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aede9e6a-6f42-4bad-b908-8057e81a5a2d admin admin] REQUEST_METHOD: `PUT` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:30:14.728843 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aede9e6a-6f42-4bad-b908-8057e81a5a2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:30:14.729260 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aede9e6a-6f42-4bad-b908-8057e81a5a2d admin admin] PATH_INFO: `/v3/projects/92b99d905bb84299b387ec16ceb58e73/users/9d36d6587bec460bb2d2549887811960/roles/9db5671c65d3414ea2682afa2ccf1403` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:30:14.747911 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aede9e6a-6f42-4bad-b908-8057e81a5a2d admin admin] RBAC: Authorizing `identity:create_grant(project_id=92b99d905bb84299b387ec16ceb58e73, user_id=9d36d6587bec460bb2d2549887811960, role_id=9db5671c65d3414ea2682afa2ccf1403)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:30:14.750619 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aede9e6a-6f42-4bad-b908-8057e81a5a2d admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:30:14.867608 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1248/2521] 10.222.0.23 () {60 vars in 1482 bytes} [Thu May 19 12:30:14 2022] PUT /identity/v3/projects/92b99d905bb84299b387ec16ceb58e73/users/9d36d6587bec460bb2d2549887811960/roles/9db5671c65d3414ea2682afa2ccf1403 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:30:14.872241 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362dbc09-704a-4bc7-b769-956d522e5a33 admin admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:30:14.883597 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362dbc09-704a-4bc7-b769-956d522e5a33 admin admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:30:14.975364 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaff2e6c-59e3-4334-b024-6a51aa7796a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:30:14.976005 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaff2e6c-59e3-4334-b024-6a51aa7796a8 admin admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:30:14.976295 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaff2e6c-59e3-4334-b024-6a51aa7796a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:30:14.976576 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaff2e6c-59e3-4334-b024-6a51aa7796a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:30:14.977353 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaff2e6c-59e3-4334-b024-6a51aa7796a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:30:14.979061 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaff2e6c-59e3-4334-b024-6a51aa7796a8 admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:30:14.991611 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1274/2522] 10.222.0.23 () {58 vars in 1121 bytes} [Thu May 19 12:30:14 2022] GET /identity/v3/roles => generated 1420 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 12:30:14.996712 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aede9e6a-6f42-4bad-b908-8057e81a5a2d admin admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:30:15.011162 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aede9e6a-6f42-4bad-b908-8057e81a5a2d admin admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:30:15.017367 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45647b9-c382-4e44-ad10-143a9771f28d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:30:15.018492 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45647b9-c382-4e44-ad10-143a9771f28d admin admin] REQUEST_METHOD: `PUT` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:30:15.019002 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45647b9-c382-4e44-ad10-143a9771f28d admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:30:15.019461 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45647b9-c382-4e44-ad10-143a9771f28d admin admin] PATH_INFO: `/v3/projects/92b99d905bb84299b387ec16ceb58e73/users/9d36d6587bec460bb2d2549887811960/roles/9db5671c65d3414ea2682afa2ccf1403` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:30:15.021969 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45647b9-c382-4e44-ad10-143a9771f28d admin admin] RBAC: Authorizing `identity:create_grant(project_id=92b99d905bb84299b387ec16ceb58e73, user_id=9d36d6587bec460bb2d2549887811960, role_id=9db5671c65d3414ea2682afa2ccf1403)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:30:15.025518 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45647b9-c382-4e44-ad10-143a9771f28d admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:30:15.040526 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1249/2523] 10.222.0.23 () {60 vars in 1482 bytes} [Thu May 19 12:30:14 2022] PUT /identity/v3/projects/92b99d905bb84299b387ec16ceb58e73/users/9d36d6587bec460bb2d2549887811960/roles/9db5671c65d3414ea2682afa2ccf1403 => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 12:30:15.059173 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e469cd7-a6fb-4eee-bf9a-08a42ef68af4 None None] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:30:15.059683 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e469cd7-a6fb-4eee-bf9a-08a42ef68af4 None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:30:15.060190 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e469cd7-a6fb-4eee-bf9a-08a42ef68af4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:30:15.096182 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.auth.core [None req-3e469cd7-a6fb-4eee-bf9a-08a42ef68af4 None None] MFA Rules not processed for user `9d36d6587bec460bb2d2549887811960`. Rule list: `[]` (Enabled: `True`). {{(pid=80540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:30:15.097444 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-3e469cd7-a6fb-4eee-bf9a-08a42ef68af4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:30:15.196806 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:30:15.204707 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1275/2524] 10.222.0.23 () {58 vars in 957 bytes} [Thu May 19 12:30:15 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:30:15.218310 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45647b9-c382-4e44-ad10-143a9771f28d admin admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:30:15.220353 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-b45647b9-c382-4e44-ad10-143a9771f28d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:30:15.321144 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45647b9-c382-4e44-ad10-143a9771f28d admin admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:30:15.325984 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffbae3e-9072-4a08-9401-c194df6a7f82 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8800254c0e784e42a791fb9abc274d79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd70d25bf69314820b8aaecb65576978a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:30:15.326699 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffbae3e-9072-4a08-9401-c194df6a7f82 service cinder] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:30:15.326985 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffbae3e-9072-4a08-9401-c194df6a7f82 service cinder] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:30:15.327270 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffbae3e-9072-4a08-9401-c194df6a7f82 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:30:15.340491 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ffbae3e-9072-4a08-9401-c194df6a7f82 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:30:15.342381 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ffbae3e-9072-4a08-9401-c194df6a7f82 service cinder] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:30:15.353979 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1250/2525] 10.222.0.23 () {60 vars in 1365 bytes} [Thu May 19 12:30:15 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 12:30:15.521816 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e469cd7-a6fb-4eee-bf9a-08a42ef68af4 None None] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:30:15.523348 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-3e469cd7-a6fb-4eee-bf9a-08a42ef68af4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:30:15.537908 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e469cd7-a6fb-4eee-bf9a-08a42ef68af4 None None] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:30:15.545546 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cee1345-cdb3-4b19-aa9b-6fec2fefd62f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8800254c0e784e42a791fb9abc274d79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd70d25bf69314820b8aaecb65576978a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:30:15.546253 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cee1345-cdb3-4b19-aa9b-6fec2fefd62f service cinder] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:30:15.546597 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cee1345-cdb3-4b19-aa9b-6fec2fefd62f service cinder] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:30:15.546901 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cee1345-cdb3-4b19-aa9b-6fec2fefd62f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:30:15.638576 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cee1345-cdb3-4b19-aa9b-6fec2fefd62f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:30:15.641376 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cee1345-cdb3-4b19-aa9b-6fec2fefd62f service cinder] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:30:15.657078 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1276/2526] 10.222.0.23 () {60 vars in 1365 bytes} [Thu May 19 12:30:15 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:30:21.629629 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffbae3e-9072-4a08-9401-c194df6a7f82 service cinder] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:30:21.759380 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffbae3e-9072-4a08-9401-c194df6a7f82 service cinder] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:30:21.768236 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b37d011-3a58-4218-8b97-c20fe52c3394 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:30:21.769827 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b37d011-3a58-4218-8b97-c20fe52c3394 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:30:21.770340 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b37d011-3a58-4218-8b97-c20fe52c3394 admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:30:21.770856 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b37d011-3a58-4218-8b97-c20fe52c3394 admin admin] PATH_INFO: `/v3/users/efce603b4efe4560b7509d32b92aa889` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:30:21.773345 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b37d011-3a58-4218-8b97-c20fe52c3394 admin admin] RBAC: Authorizing `identity:delete_user(user_id=efce603b4efe4560b7509d32b92aa889)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:30:21.778519 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b37d011-3a58-4218-8b97-c20fe52c3394 admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:30:21.822026 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=94,keystone:INSERT=5,keystone:DELETE=1 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:30:22.020157 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-3b37d011-3a58-4218-8b97-c20fe52c3394 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'efce603b4efe4560b7509d32b92aa889'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:30:22.037850 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-3b37d011-3a58-4218-8b97-c20fe52c3394 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'efce603b4efe4560b7509d32b92aa889'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:30:22.044965 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-3b37d011-3a58-4218-8b97-c20fe52c3394 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'efce603b4efe4560b7509d32b92aa889'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:30:22.152110 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-3b37d011-3a58-4218-8b97-c20fe52c3394 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'efce603b4efe4560b7509d32b92aa889'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:30:22.161005 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-3b37d011-3a58-4218-8b97-c20fe52c3394 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'efce603b4efe4560b7509d32b92aa889'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:30:22.169153 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1251/2527] 10.222.0.23 () {58 vars in 1223 bytes} [Thu May 19 12:30:21 2022] DELETE /identity/v3/users/efce603b4efe4560b7509d32b92aa889 => generated 0 bytes in 541 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:30:22.174106 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4fd662-32de-4d20-a015-6c62ddd70813 None None] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:30:22.174790 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4fd662-32de-4d20-a015-6c62ddd70813 None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:30:22.174790 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4fd662-32de-4d20-a015-6c62ddd70813 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:30:22.195347 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.auth.core [None req-6f4fd662-32de-4d20-a015-6c62ddd70813 None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:30:22.196619 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-6f4fd662-32de-4d20-a015-6c62ddd70813 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:30:22.289441 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1277/2528] 10.222.0.23 () {58 vars in 957 bytes} [Thu May 19 12:30:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:30:22.305806 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b37d011-3a58-4218-8b97-c20fe52c3394 admin admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:30:22.309248 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-3b37d011-3a58-4218-8b97-c20fe52c3394 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:30:22.413890 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b37d011-3a58-4218-8b97-c20fe52c3394 admin admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:30:22.418816 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65735b99-18a1-4d83-8940-b9cdeea257d0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a00b3fa7497456895e5e0bf105c4fc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd70d25bf69314820b8aaecb65576978a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:30:22.419502 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65735b99-18a1-4d83-8940-b9cdeea257d0 service neutron] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:30:22.419861 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65735b99-18a1-4d83-8940-b9cdeea257d0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:30:22.433004 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65735b99-18a1-4d83-8940-b9cdeea257d0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:30:22.433004 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65735b99-18a1-4d83-8940-b9cdeea257d0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:30:22.433004 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65735b99-18a1-4d83-8940-b9cdeea257d0 service neutron] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:30:22.443284 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1252/2529] 10.222.0.23 () {60 vars in 1365 bytes} [Thu May 19 12:30:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:30:22.590084 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4fd662-32de-4d20-a015-6c62ddd70813 None None] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:30:22.592798 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-6f4fd662-32de-4d20-a015-6c62ddd70813 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:30:22.825020 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4fd662-32de-4d20-a015-6c62ddd70813 None None] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:30:22.856857 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106916e8-084a-4177-962c-a244ffb4559b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34a98d75d84b4c67981377e0f26eca62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd70d25bf69314820b8aaecb65576978a', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:30:22.865040 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106916e8-084a-4177-962c-a244ffb4559b service nova] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:30:22.865660 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106916e8-084a-4177-962c-a244ffb4559b service nova] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:30:22.866302 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106916e8-084a-4177-962c-a244ffb4559b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:30:22.879954 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-106916e8-084a-4177-962c-a244ffb4559b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:30:22.926416 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:30:22.930614 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106916e8-084a-4177-962c-a244ffb4559b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:30:22.932130 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106916e8-084a-4177-962c-a244ffb4559b service nova] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:30:22.947359 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1278/2530] 10.222.0.23 () {60 vars in 1365 bytes} [Thu May 19 12:30:22 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:30:23.127544 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65735b99-18a1-4d83-8940-b9cdeea257d0 service neutron] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:30:23.131223 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-65735b99-18a1-4d83-8940-b9cdeea257d0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:30:23.156877 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65735b99-18a1-4d83-8940-b9cdeea257d0 service neutron] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:30:23.167543 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c83fbc0-b4f4-42b6-ba0f-e10eecb2753e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:30:23.169611 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c83fbc0-b4f4-42b6-ba0f-e10eecb2753e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:30:23.170056 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c83fbc0-b4f4-42b6-ba0f-e10eecb2753e admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:30:23.170791 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c83fbc0-b4f4-42b6-ba0f-e10eecb2753e admin admin] PATH_INFO: `/v3/projects/e03eae01937440af9454620be039539a` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:30:23.175735 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c83fbc0-b4f4-42b6-ba0f-e10eecb2753e admin admin] RBAC: Authorizing `identity:delete_project(project_id=e03eae01937440af9454620be039539a)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:30:23.182720 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c83fbc0-b4f4-42b6-ba0f-e10eecb2753e admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:30:23.344906 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-9c83fbc0-b4f4-42b6-ba0f-e10eecb2753e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e03eae01937440af9454620be039539a', 'request_id': 'req-9c83fbc0-b4f4-42b6-ba0f-e10eecb2753e'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:30:23.353913 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-9c83fbc0-b4f4-42b6-ba0f-e10eecb2753e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e03eae01937440af9454620be039539a', 'request_id': 'req-9c83fbc0-b4f4-42b6-ba0f-e10eecb2753e'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:30:23.477151 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-9c83fbc0-b4f4-42b6-ba0f-e10eecb2753e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e03eae01937440af9454620be039539a', 'request_id': 'req-9c83fbc0-b4f4-42b6-ba0f-e10eecb2753e'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:30:23.500112 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-9c83fbc0-b4f4-42b6-ba0f-e10eecb2753e admin admin] The token cache is being invalidate because project e03eae01937440af9454620be039539a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80539) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 12:30:23.500587 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-9c83fbc0-b4f4-42b6-ba0f-e10eecb2753e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:30:23.501542 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-9c83fbc0-b4f4-42b6-ba0f-e10eecb2753e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:30:23.503210 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1253/2531] 10.222.0.23 () {58 vars in 1232 bytes} [Thu May 19 12:30:23 2022] DELETE /identity/v3/projects/e03eae01937440af9454620be039539a => generated 0 bytes in 379 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:30:23.507637 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106916e8-084a-4177-962c-a244ffb4559b service nova] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:30:23.518791 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106916e8-084a-4177-962c-a244ffb4559b service nova] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:30:23.619265 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de91fff5-8963-4030-832f-351a0ccf61eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:30:23.621162 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de91fff5-8963-4030-832f-351a0ccf61eb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:30:23.621641 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de91fff5-8963-4030-832f-351a0ccf61eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:30:23.621977 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de91fff5-8963-4030-832f-351a0ccf61eb admin admin] PATH_INFO: `/v3/users/9d36d6587bec460bb2d2549887811960` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:30:23.623990 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de91fff5-8963-4030-832f-351a0ccf61eb admin admin] RBAC: Authorizing `identity:delete_user(user_id=9d36d6587bec460bb2d2549887811960)` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:30:23.626322 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de91fff5-8963-4030-832f-351a0ccf61eb admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:30:23.956221 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-de91fff5-8963-4030-832f-351a0ccf61eb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9d36d6587bec460bb2d2549887811960'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:30:23.987843 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-de91fff5-8963-4030-832f-351a0ccf61eb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9d36d6587bec460bb2d2549887811960'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:30:23.989186 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-de91fff5-8963-4030-832f-351a0ccf61eb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9d36d6587bec460bb2d2549887811960'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:30:24.237033 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-de91fff5-8963-4030-832f-351a0ccf61eb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9d36d6587bec460bb2d2549887811960'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:30:24.248074 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-de91fff5-8963-4030-832f-351a0ccf61eb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9d36d6587bec460bb2d2549887811960'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:30:24.255077 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1279/2532] 10.222.0.23 () {58 vars in 1223 bytes} [Thu May 19 12:30:23 2022] DELETE /identity/v3/users/9d36d6587bec460bb2d2549887811960 => generated 0 bytes in 748 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 12:30:24.835462 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c83fbc0-b4f4-42b6-ba0f-e10eecb2753e admin admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:30:24.835462 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-9c83fbc0-b4f4-42b6-ba0f-e10eecb2753e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:30:24.851664 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c83fbc0-b4f4-42b6-ba0f-e10eecb2753e admin admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:30:24.872836 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:30:24.949561 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9194d1-3e77-40cd-9e75-fc68385101ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:30:24.951091 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9194d1-3e77-40cd-9e75-fc68385101ea admin admin] REQUEST_METHOD: `DELETE` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:30:24.951566 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9194d1-3e77-40cd-9e75-fc68385101ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:30:24.952092 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9194d1-3e77-40cd-9e75-fc68385101ea admin admin] PATH_INFO: `/v3/projects/92b99d905bb84299b387ec16ceb58e73` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:30:24.956627 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9194d1-3e77-40cd-9e75-fc68385101ea admin admin] RBAC: Authorizing `identity:delete_project(project_id=92b99d905bb84299b387ec16ceb58e73)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:30:24.960453 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9194d1-3e77-40cd-9e75-fc68385101ea admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:30:25.191549 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-cf9194d1-3e77-40cd-9e75-fc68385101ea admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '92b99d905bb84299b387ec16ceb58e73', 'request_id': 'req-cf9194d1-3e77-40cd-9e75-fc68385101ea'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:30:25.198452 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-cf9194d1-3e77-40cd-9e75-fc68385101ea admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '92b99d905bb84299b387ec16ceb58e73', 'request_id': 'req-cf9194d1-3e77-40cd-9e75-fc68385101ea'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:30:25.282980 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-cf9194d1-3e77-40cd-9e75-fc68385101ea admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '92b99d905bb84299b387ec16ceb58e73', 'request_id': 'req-cf9194d1-3e77-40cd-9e75-fc68385101ea'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:30:25.297114 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-cf9194d1-3e77-40cd-9e75-fc68385101ea admin admin] The token cache is being invalidate because project 92b99d905bb84299b387ec16ceb58e73 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80539) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 12:30:25.300094 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-cf9194d1-3e77-40cd-9e75-fc68385101ea admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:30:25.300094 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-cf9194d1-3e77-40cd-9e75-fc68385101ea admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:30:25.300094 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1254/2533] 10.222.0.23 () {58 vars in 1232 bytes} [Thu May 19 12:30:24 2022] DELETE /identity/v3/projects/92b99d905bb84299b387ec16ceb58e73 => generated 0 bytes in 472 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 12:30:25.304933 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600e8ccf-4bc3-42b7-8335-410d9e9decc8 None None] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:30:25.305470 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600e8ccf-4bc3-42b7-8335-410d9e9decc8 None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:30:25.305600 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600e8ccf-4bc3-42b7-8335-410d9e9decc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:30:25.326057 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.auth.core [None req-600e8ccf-4bc3-42b7-8335-410d9e9decc8 None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:30:25.327360 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-600e8ccf-4bc3-42b7-8335-410d9e9decc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:30:25.414651 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1280/2534] 10.222.0.23 () {58 vars in 957 bytes} [Thu May 19 12:30:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 12:30:25.423552 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beec6a87-4458-4daa-922a-31845713252d None None] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:30:25.423878 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beec6a87-4458-4daa-922a-31845713252d None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:30:25.424164 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beec6a87-4458-4daa-922a-31845713252d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:30:25.444282 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.auth.core [None req-beec6a87-4458-4daa-922a-31845713252d None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:30:25.445493 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-beec6a87-4458-4daa-922a-31845713252d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:30:25.454219 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1255/2535] 10.222.0.23 () {58 vars in 957 bytes} [Thu May 19 12:30:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 12:30:25.459214 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600e8ccf-4bc3-42b7-8335-410d9e9decc8 None None] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:30:25.472519 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600e8ccf-4bc3-42b7-8335-410d9e9decc8 None None] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:30:25.477985 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b16dc2e-b710-44da-90c7-bfd687b1c4bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:30:25.485239 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b16dc2e-b710-44da-90c7-bfd687b1c4bc admin admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:30:25.485239 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b16dc2e-b710-44da-90c7-bfd687b1c4bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:30:25.485239 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b16dc2e-b710-44da-90c7-bfd687b1c4bc admin admin] PATH_INFO: `/v3/domains` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:30:25.485239 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b16dc2e-b710-44da-90c7-bfd687b1c4bc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:30:25.485239 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b16dc2e-b710-44da-90c7-bfd687b1c4bc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:30:25.485239 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b16dc2e-b710-44da-90c7-bfd687b1c4bc admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:30:25.497672 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1281/2536] 10.222.0.23 () {58 vars in 1152 bytes} [Thu May 19 12:30:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 12:30:25.503524 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beec6a87-4458-4daa-922a-31845713252d None None] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:30:25.519415 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beec6a87-4458-4daa-922a-31845713252d None None] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:30:25.524957 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b362c4-d0be-436a-8ec6-6df7f117c465 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:30:25.525634 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b362c4-d0be-436a-8ec6-6df7f117c465 admin admin] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:30:25.526437 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b362c4-d0be-436a-8ec6-6df7f117c465 admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:30:25.526786 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b362c4-d0be-436a-8ec6-6df7f117c465 admin admin] PATH_INFO: `/v3/projects` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:30:25.527567 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b362c4-d0be-436a-8ec6-6df7f117c465 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:30:25.529577 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b362c4-d0be-436a-8ec6-6df7f117c465 admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:30:25.666096 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1256/2537] 10.222.0.23 () {60 vars in 1152 bytes} [Thu May 19 12:30:25 2022] POST /identity/v3/projects => generated 387 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 12:30:25.672302 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b16dc2e-b710-44da-90c7-bfd687b1c4bc admin admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:30:25.679148 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:30:25.691424 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b16dc2e-b710-44da-90c7-bfd687b1c4bc admin admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:30:25.763404 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c49afe5-352a-43ac-803a-cea15d1fa557 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:30:25.764248 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c49afe5-352a-43ac-803a-cea15d1fa557 admin admin] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:30:25.764389 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c49afe5-352a-43ac-803a-cea15d1fa557 admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:30:25.764721 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c49afe5-352a-43ac-803a-cea15d1fa557 admin admin] PATH_INFO: `/v3/users` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:30:25.765933 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c49afe5-352a-43ac-803a-cea15d1fa557 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:30:25.768108 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c49afe5-352a-43ac-803a-cea15d1fa557 admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:30:25.895542 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1282/2538] 10.222.0.23 () {60 vars in 1143 bytes} [Thu May 19 12:30:25 2022] POST /identity/v3/users => generated 294 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 12:30:25.900250 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b362c4-d0be-436a-8ec6-6df7f117c465 admin admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:30:25.915974 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b362c4-d0be-436a-8ec6-6df7f117c465 admin admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:30:25.921560 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d941c34-62ff-43bf-a134-6c7a8f7fc1b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:30:25.932899 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d941c34-62ff-43bf-a134-6c7a8f7fc1b6 admin admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:30:25.932899 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d941c34-62ff-43bf-a134-6c7a8f7fc1b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:30:25.932899 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d941c34-62ff-43bf-a134-6c7a8f7fc1b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:30:25.932899 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d941c34-62ff-43bf-a134-6c7a8f7fc1b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:30:25.932899 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d941c34-62ff-43bf-a134-6c7a8f7fc1b6 admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:30:25.940672 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1257/2539] 10.222.0.23 () {58 vars in 1121 bytes} [Thu May 19 12:30:25 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 12:30:25.946186 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c49afe5-352a-43ac-803a-cea15d1fa557 admin admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:30:25.958120 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c49afe5-352a-43ac-803a-cea15d1fa557 admin admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:30:25.964073 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5852d0ab-478e-4910-beca-fa517d979db7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:30:25.965054 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5852d0ab-478e-4910-beca-fa517d979db7 admin admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:30:25.965388 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5852d0ab-478e-4910-beca-fa517d979db7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:30:25.965713 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5852d0ab-478e-4910-beca-fa517d979db7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:30:25.966515 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5852d0ab-478e-4910-beca-fa517d979db7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:30:25.968554 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5852d0ab-478e-4910-beca-fa517d979db7 admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:30:25.982638 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1283/2540] 10.222.0.23 () {58 vars in 1121 bytes} [Thu May 19 12:30:25 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 12:30:25.988015 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d941c34-62ff-43bf-a134-6c7a8f7fc1b6 admin admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:30:26.002090 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d941c34-62ff-43bf-a134-6c7a8f7fc1b6 admin admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:30:26.008417 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb48ad57-762e-46a2-b4ef-fd32cf3a7dc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:30:26.009470 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb48ad57-762e-46a2-b4ef-fd32cf3a7dc6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:30:26.009877 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb48ad57-762e-46a2-b4ef-fd32cf3a7dc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:30:26.010944 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb48ad57-762e-46a2-b4ef-fd32cf3a7dc6 admin admin] PATH_INFO: `/v3/projects/2041411f6239443893ee58c7353c0813/users/928e8907d624406f8155af5c3416278a/roles/259eb400cdb4439db72d14456ae9cd08` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:30:26.039321 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb48ad57-762e-46a2-b4ef-fd32cf3a7dc6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2041411f6239443893ee58c7353c0813, user_id=928e8907d624406f8155af5c3416278a, role_id=259eb400cdb4439db72d14456ae9cd08)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:30:26.044904 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb48ad57-762e-46a2-b4ef-fd32cf3a7dc6 admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:30:26.184402 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1258/2541] 10.222.0.23 () {60 vars in 1482 bytes} [Thu May 19 12:30:25 2022] PUT /identity/v3/projects/2041411f6239443893ee58c7353c0813/users/928e8907d624406f8155af5c3416278a/roles/259eb400cdb4439db72d14456ae9cd08 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:30:26.197527 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276aedd8-4bea-44fd-8c2c-7a9b5820ccef None None] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:30:26.198496 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276aedd8-4bea-44fd-8c2c-7a9b5820ccef None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:30:26.198911 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276aedd8-4bea-44fd-8c2c-7a9b5820ccef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:30:26.231210 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.auth.core [None req-276aedd8-4bea-44fd-8c2c-7a9b5820ccef None None] MFA Rules not processed for user `928e8907d624406f8155af5c3416278a`. Rule list: `[]` (Enabled: `True`). {{(pid=80540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:30:26.232484 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-276aedd8-4bea-44fd-8c2c-7a9b5820ccef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:30:26.315273 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1284/2542] 10.222.0.23 () {58 vars in 957 bytes} [Thu May 19 12:30:26 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 12:30:26.320579 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb48ad57-762e-46a2-b4ef-fd32cf3a7dc6 admin admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:30:26.377343 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:30:26.396909 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb48ad57-762e-46a2-b4ef-fd32cf3a7dc6 admin admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:30:26.402698 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ed4ee9-9583-48de-bbdc-239efe407ec5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:30:26.403384 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ed4ee9-9583-48de-bbdc-239efe407ec5 admin admin] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:30:26.403672 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ed4ee9-9583-48de-bbdc-239efe407ec5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:30:26.404005 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ed4ee9-9583-48de-bbdc-239efe407ec5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:30:26.404720 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ed4ee9-9583-48de-bbdc-239efe407ec5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:30:26.406924 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ed4ee9-9583-48de-bbdc-239efe407ec5 admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:30:26.559834 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1259/2543] 10.222.0.23 () {60 vars in 1152 bytes} [Thu May 19 12:30:26 2022] POST /identity/v3/projects => generated 387 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 12:30:26.566410 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276aedd8-4bea-44fd-8c2c-7a9b5820ccef None None] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:30:26.580956 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276aedd8-4bea-44fd-8c2c-7a9b5820ccef None None] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:30:26.648288 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590274c8-93e2-48e6-ba87-8b64847b3e6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:30:26.648975 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590274c8-93e2-48e6-ba87-8b64847b3e6e admin admin] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:30:26.649286 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590274c8-93e2-48e6-ba87-8b64847b3e6e admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:30:26.649567 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590274c8-93e2-48e6-ba87-8b64847b3e6e admin admin] PATH_INFO: `/v3/users` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:30:26.650284 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590274c8-93e2-48e6-ba87-8b64847b3e6e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:30:26.653703 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590274c8-93e2-48e6-ba87-8b64847b3e6e admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:30:26.764853 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:30:26.772135 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1285/2544] 10.222.0.23 () {60 vars in 1143 bytes} [Thu May 19 12:30:26 2022] POST /identity/v3/users => generated 300 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 12:30:26.788016 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ed4ee9-9583-48de-bbdc-239efe407ec5 admin admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:30:26.800825 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ed4ee9-9583-48de-bbdc-239efe407ec5 admin admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:30:26.810990 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ae4f9b-769e-405d-aa2e-aa81b56c416f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:30:26.812274 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ae4f9b-769e-405d-aa2e-aa81b56c416f admin admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:30:26.812835 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ae4f9b-769e-405d-aa2e-aa81b56c416f admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:30:26.813378 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ae4f9b-769e-405d-aa2e-aa81b56c416f admin admin] PATH_INFO: `/v3/roles` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:30:26.815610 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ae4f9b-769e-405d-aa2e-aa81b56c416f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:30:26.818943 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ae4f9b-769e-405d-aa2e-aa81b56c416f admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:30:26.839197 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1260/2545] 10.222.0.23 () {58 vars in 1121 bytes} [Thu May 19 12:30:26 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 12:30:26.840755 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590274c8-93e2-48e6-ba87-8b64847b3e6e admin admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:30:26.854805 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590274c8-93e2-48e6-ba87-8b64847b3e6e admin admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:30:26.866162 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad4f8f6-e20c-41bd-a5a7-df333ddf89fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:30:26.866162 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad4f8f6-e20c-41bd-a5a7-df333ddf89fc admin admin] REQUEST_METHOD: `PUT` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:30:26.866162 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad4f8f6-e20c-41bd-a5a7-df333ddf89fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:30:26.866162 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad4f8f6-e20c-41bd-a5a7-df333ddf89fc admin admin] PATH_INFO: `/v3/projects/81a5ab3f3abf484b9372fe91256fd40c/users/63d6774415ad4a1d8459c855ab8a34e3/roles/9db5671c65d3414ea2682afa2ccf1403` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:30:26.892069 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad4f8f6-e20c-41bd-a5a7-df333ddf89fc admin admin] RBAC: Authorizing `identity:create_grant(project_id=81a5ab3f3abf484b9372fe91256fd40c, user_id=63d6774415ad4a1d8459c855ab8a34e3, role_id=9db5671c65d3414ea2682afa2ccf1403)` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:30:26.896094 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad4f8f6-e20c-41bd-a5a7-df333ddf89fc admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:30:27.034518 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1286/2546] 10.222.0.23 () {60 vars in 1482 bytes} [Thu May 19 12:30:26 2022] PUT /identity/v3/projects/81a5ab3f3abf484b9372fe91256fd40c/users/63d6774415ad4a1d8459c855ab8a34e3/roles/9db5671c65d3414ea2682afa2ccf1403 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 12:30:27.038797 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ae4f9b-769e-405d-aa2e-aa81b56c416f admin admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:30:27.050590 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ae4f9b-769e-405d-aa2e-aa81b56c416f admin admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:30:27.133796 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5551cd1-5edd-421d-8df2-de7eb5a369ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:30:27.134475 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5551cd1-5edd-421d-8df2-de7eb5a369ee admin admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:30:27.134847 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5551cd1-5edd-421d-8df2-de7eb5a369ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:30:27.135767 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5551cd1-5edd-421d-8df2-de7eb5a369ee admin admin] PATH_INFO: `/v3/roles` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:30:27.138689 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5551cd1-5edd-421d-8df2-de7eb5a369ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:30:27.145027 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5551cd1-5edd-421d-8df2-de7eb5a369ee admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:30:27.161131 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1261/2547] 10.222.0.23 () {58 vars in 1121 bytes} [Thu May 19 12:30:27 2022] GET /identity/v3/roles => generated 1420 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 19 12:30:27.165807 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad4f8f6-e20c-41bd-a5a7-df333ddf89fc admin admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:30:27.196281 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad4f8f6-e20c-41bd-a5a7-df333ddf89fc admin admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:30:27.196281 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d8091a-f1e9-448f-876f-37207c9f2717 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:30:27.196281 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d8091a-f1e9-448f-876f-37207c9f2717 admin admin] REQUEST_METHOD: `PUT` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:30:27.196281 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d8091a-f1e9-448f-876f-37207c9f2717 admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:30:27.196945 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d8091a-f1e9-448f-876f-37207c9f2717 admin admin] PATH_INFO: `/v3/projects/81a5ab3f3abf484b9372fe91256fd40c/users/63d6774415ad4a1d8459c855ab8a34e3/roles/9db5671c65d3414ea2682afa2ccf1403` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:30:27.201940 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d8091a-f1e9-448f-876f-37207c9f2717 admin admin] RBAC: Authorizing `identity:create_grant(project_id=81a5ab3f3abf484b9372fe91256fd40c, user_id=63d6774415ad4a1d8459c855ab8a34e3, role_id=9db5671c65d3414ea2682afa2ccf1403)` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:30:27.205867 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d8091a-f1e9-448f-876f-37207c9f2717 admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:30:27.228409 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1287/2548] 10.222.0.23 () {60 vars in 1482 bytes} [Thu May 19 12:30:27 2022] PUT /identity/v3/projects/81a5ab3f3abf484b9372fe91256fd40c/users/63d6774415ad4a1d8459c855ab8a34e3/roles/9db5671c65d3414ea2682afa2ccf1403 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:30:27.236364 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f09525-4bd3-4350-8a6e-5bbd02039f55 None None] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:30:27.237246 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f09525-4bd3-4350-8a6e-5bbd02039f55 None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:30:27.237566 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f09525-4bd3-4350-8a6e-5bbd02039f55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:30:27.274860 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.auth.core [None req-78f09525-4bd3-4350-8a6e-5bbd02039f55 None None] MFA Rules not processed for user `63d6774415ad4a1d8459c855ab8a34e3`. Rule list: `[]` (Enabled: `True`). {{(pid=80539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:30:27.276861 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-78f09525-4bd3-4350-8a6e-5bbd02039f55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:30:27.356717 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1262/2549] 10.222.0.23 () {58 vars in 957 bytes} [Thu May 19 12:30:27 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:30:27.368951 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d8091a-f1e9-448f-876f-37207c9f2717 admin admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:30:27.370920 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-24d8091a-f1e9-448f-876f-37207c9f2717 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:30:27.439609 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d8091a-f1e9-448f-876f-37207c9f2717 admin admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:30:27.444426 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33742c2c-3ab4-4749-965c-ba0c6fcc38f8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8800254c0e784e42a791fb9abc274d79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd70d25bf69314820b8aaecb65576978a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:30:27.445235 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33742c2c-3ab4-4749-965c-ba0c6fcc38f8 service cinder] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:30:27.445619 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33742c2c-3ab4-4749-965c-ba0c6fcc38f8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:30:27.445975 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33742c2c-3ab4-4749-965c-ba0c6fcc38f8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:30:27.457512 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33742c2c-3ab4-4749-965c-ba0c6fcc38f8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:30:27.458970 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33742c2c-3ab4-4749-965c-ba0c6fcc38f8 service cinder] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:30:27.470384 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1288/2550] 10.222.0.23 () {60 vars in 1365 bytes} [Thu May 19 12:30:27 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 12:30:27.502404 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f09525-4bd3-4350-8a6e-5bbd02039f55 None None] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:30:27.557661 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:30:27.585797 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f09525-4bd3-4350-8a6e-5bbd02039f55 None None] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:30:27.591691 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0962414d-8297-4a66-baca-f3545f0880c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:30:27.592719 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0962414d-8297-4a66-baca-f3545f0880c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:30:27.593074 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0962414d-8297-4a66-baca-f3545f0880c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:30:27.593394 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0962414d-8297-4a66-baca-f3545f0880c3 admin admin] PATH_INFO: `/v3/users/928e8907d624406f8155af5c3416278a` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:30:27.594784 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0962414d-8297-4a66-baca-f3545f0880c3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=928e8907d624406f8155af5c3416278a)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:30:27.597562 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0962414d-8297-4a66-baca-f3545f0880c3 admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:30:27.842377 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-0962414d-8297-4a66-baca-f3545f0880c3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '928e8907d624406f8155af5c3416278a'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:30:27.858466 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-0962414d-8297-4a66-baca-f3545f0880c3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '928e8907d624406f8155af5c3416278a'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:30:27.859228 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-0962414d-8297-4a66-baca-f3545f0880c3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '928e8907d624406f8155af5c3416278a'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:30:27.982330 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-0962414d-8297-4a66-baca-f3545f0880c3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '928e8907d624406f8155af5c3416278a'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:30:27.999692 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-0962414d-8297-4a66-baca-f3545f0880c3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '928e8907d624406f8155af5c3416278a'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:30:28.009977 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1263/2551] 10.222.0.23 () {58 vars in 1223 bytes} [Thu May 19 12:30:27 2022] DELETE /identity/v3/users/928e8907d624406f8155af5c3416278a => generated 0 bytes in 508 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 12:30:28.015364 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9733d874-ff29-4ce9-9b80-ea0c1e57dd6d None None] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:30:28.015836 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9733d874-ff29-4ce9-9b80-ea0c1e57dd6d None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:30:28.016217 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9733d874-ff29-4ce9-9b80-ea0c1e57dd6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:30:28.037694 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.auth.core [None req-9733d874-ff29-4ce9-9b80-ea0c1e57dd6d None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:30:28.038937 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-9733d874-ff29-4ce9-9b80-ea0c1e57dd6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:30:28.109313 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1289/2552] 10.222.0.23 () {58 vars in 957 bytes} [Thu May 19 12:30:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:30:28.120176 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0962414d-8297-4a66-baca-f3545f0880c3 admin admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:30:28.122085 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-0962414d-8297-4a66-baca-f3545f0880c3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:30:28.188603 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0962414d-8297-4a66-baca-f3545f0880c3 admin admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:30:28.192603 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799231e1-1aa7-46e8-88bf-604754e0de02 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a00b3fa7497456895e5e0bf105c4fc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd70d25bf69314820b8aaecb65576978a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:30:28.193396 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799231e1-1aa7-46e8-88bf-604754e0de02 service neutron] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:30:28.193725 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799231e1-1aa7-46e8-88bf-604754e0de02 service neutron] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:30:28.194045 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799231e1-1aa7-46e8-88bf-604754e0de02 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:30:28.205667 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799231e1-1aa7-46e8-88bf-604754e0de02 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:30:28.207617 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799231e1-1aa7-46e8-88bf-604754e0de02 service neutron] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:30:28.222337 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1264/2553] 10.222.0.23 () {60 vars in 1365 bytes} [Thu May 19 12:30:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:30:28.796267 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9733d874-ff29-4ce9-9b80-ea0c1e57dd6d None None] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:30:28.798456 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-9733d874-ff29-4ce9-9b80-ea0c1e57dd6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:30:28.827863 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9733d874-ff29-4ce9-9b80-ea0c1e57dd6d None None] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:30:28.836017 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7306464f-3c6c-4ae0-aab1-35467b092583 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:30:28.837552 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7306464f-3c6c-4ae0-aab1-35467b092583 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:30:28.838106 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7306464f-3c6c-4ae0-aab1-35467b092583 admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:30:28.839223 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7306464f-3c6c-4ae0-aab1-35467b092583 admin admin] PATH_INFO: `/v3/projects/2041411f6239443893ee58c7353c0813` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:30:28.840881 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7306464f-3c6c-4ae0-aab1-35467b092583 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2041411f6239443893ee58c7353c0813)` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:30:28.842944 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7306464f-3c6c-4ae0-aab1-35467b092583 admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:30:29.012817 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-7306464f-3c6c-4ae0-aab1-35467b092583 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2041411f6239443893ee58c7353c0813', 'request_id': 'req-7306464f-3c6c-4ae0-aab1-35467b092583'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:30:29.019550 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-7306464f-3c6c-4ae0-aab1-35467b092583 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2041411f6239443893ee58c7353c0813', 'request_id': 'req-7306464f-3c6c-4ae0-aab1-35467b092583'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:30:29.024687 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=92,keystone:INSERT=2,keystone:DELETE=6 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:30:29.025949 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-7306464f-3c6c-4ae0-aab1-35467b092583 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2041411f6239443893ee58c7353c0813', 'request_id': 'req-7306464f-3c6c-4ae0-aab1-35467b092583'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:30:29.113119 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-7306464f-3c6c-4ae0-aab1-35467b092583 admin admin] The token cache is being invalidate because project 2041411f6239443893ee58c7353c0813 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80540) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 12:30:29.113525 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-7306464f-3c6c-4ae0-aab1-35467b092583 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:30:29.114379 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-7306464f-3c6c-4ae0-aab1-35467b092583 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:30:29.116406 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1290/2554] 10.222.0.23 () {58 vars in 1232 bytes} [Thu May 19 12:30:28 2022] DELETE /identity/v3/projects/2041411f6239443893ee58c7353c0813 => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:30:29.122062 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799231e1-1aa7-46e8-88bf-604754e0de02 service neutron] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:30:29.141050 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799231e1-1aa7-46e8-88bf-604754e0de02 service neutron] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:30:29.233354 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0277b9-b829-46c6-9e28-9a14e7e48f4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:30:29.234147 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0277b9-b829-46c6-9e28-9a14e7e48f4a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:30:29.234432 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0277b9-b829-46c6-9e28-9a14e7e48f4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:30:29.234715 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0277b9-b829-46c6-9e28-9a14e7e48f4a admin admin] PATH_INFO: `/v3/users/63d6774415ad4a1d8459c855ab8a34e3` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:30:29.236127 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0277b9-b829-46c6-9e28-9a14e7e48f4a admin admin] RBAC: Authorizing `identity:delete_user(user_id=63d6774415ad4a1d8459c855ab8a34e3)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:30:29.238337 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0277b9-b829-46c6-9e28-9a14e7e48f4a admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:30:29.240131 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:30:29.515098 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-6d0277b9-b829-46c6-9e28-9a14e7e48f4a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '63d6774415ad4a1d8459c855ab8a34e3'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:30:29.528884 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-6d0277b9-b829-46c6-9e28-9a14e7e48f4a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '63d6774415ad4a1d8459c855ab8a34e3'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:30:29.529661 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-6d0277b9-b829-46c6-9e28-9a14e7e48f4a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '63d6774415ad4a1d8459c855ab8a34e3'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:30:29.622077 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-6d0277b9-b829-46c6-9e28-9a14e7e48f4a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '63d6774415ad4a1d8459c855ab8a34e3'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:30:29.630245 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-6d0277b9-b829-46c6-9e28-9a14e7e48f4a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '63d6774415ad4a1d8459c855ab8a34e3'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:30:29.637007 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1265/2555] 10.222.0.23 () {58 vars in 1223 bytes} [Thu May 19 12:30:29 2022] DELETE /identity/v3/users/63d6774415ad4a1d8459c855ab8a34e3 => generated 0 bytes in 516 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:30:29.920038 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7306464f-3c6c-4ae0-aab1-35467b092583 admin admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:30:29.922092 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-7306464f-3c6c-4ae0-aab1-35467b092583 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:30:29.992743 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7306464f-3c6c-4ae0-aab1-35467b092583 admin admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:30:30.062400 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e5442f-4c4a-4df1-949a-79447c078bc5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a00b3fa7497456895e5e0bf105c4fc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd70d25bf69314820b8aaecb65576978a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:30:30.063697 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e5442f-4c4a-4df1-949a-79447c078bc5 service neutron] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:30:30.064283 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e5442f-4c4a-4df1-949a-79447c078bc5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:30:30.064793 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e5442f-4c4a-4df1-949a-79447c078bc5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:30:30.068130 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-15e5442f-4c4a-4df1-949a-79447c078bc5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:30:30.154332 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e5442f-4c4a-4df1-949a-79447c078bc5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:30:30.155898 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e5442f-4c4a-4df1-949a-79447c078bc5 service neutron] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:30:30.169200 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1291/2556] 10.222.0.23 () {60 vars in 1365 bytes} [Thu May 19 12:30:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 250 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:30:30.326071 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0277b9-b829-46c6-9e28-9a14e7e48f4a admin admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:30:30.327921 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-6d0277b9-b829-46c6-9e28-9a14e7e48f4a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:30:30.362884 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0277b9-b829-46c6-9e28-9a14e7e48f4a admin admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:30:30.368891 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730484a2-04d5-4119-b860-429c276338de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:30:30.370509 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730484a2-04d5-4119-b860-429c276338de admin admin] REQUEST_METHOD: `DELETE` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:30:30.371056 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730484a2-04d5-4119-b860-429c276338de admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:30:30.371340 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730484a2-04d5-4119-b860-429c276338de admin admin] PATH_INFO: `/v3/projects/81a5ab3f3abf484b9372fe91256fd40c` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:30:30.373033 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730484a2-04d5-4119-b860-429c276338de admin admin] RBAC: Authorizing `identity:delete_project(project_id=81a5ab3f3abf484b9372fe91256fd40c)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:30:30.375209 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730484a2-04d5-4119-b860-429c276338de admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:30:30.530369 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-730484a2-04d5-4119-b860-429c276338de admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '81a5ab3f3abf484b9372fe91256fd40c', 'request_id': 'req-730484a2-04d5-4119-b860-429c276338de'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:30:30.537713 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-730484a2-04d5-4119-b860-429c276338de admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '81a5ab3f3abf484b9372fe91256fd40c', 'request_id': 'req-730484a2-04d5-4119-b860-429c276338de'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:30:30.654621 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-730484a2-04d5-4119-b860-429c276338de admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '81a5ab3f3abf484b9372fe91256fd40c', 'request_id': 'req-730484a2-04d5-4119-b860-429c276338de'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:30:30.671048 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-730484a2-04d5-4119-b860-429c276338de admin admin] The token cache is being invalidate because project 81a5ab3f3abf484b9372fe91256fd40c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80539) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 12:30:30.676767 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-730484a2-04d5-4119-b860-429c276338de admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:30:30.676767 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-730484a2-04d5-4119-b860-429c276338de admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:30:30.676767 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1266/2557] 10.222.0.23 () {58 vars in 1232 bytes} [Thu May 19 12:30:30 2022] DELETE /identity/v3/projects/81a5ab3f3abf484b9372fe91256fd40c => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:30:30.680278 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89a6f20-e3a3-40a5-8d56-b844e13df510 None None] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:30:30.680790 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89a6f20-e3a3-40a5-8d56-b844e13df510 None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:30:30.681296 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89a6f20-e3a3-40a5-8d56-b844e13df510 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:30:30.710952 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.auth.core [None req-c89a6f20-e3a3-40a5-8d56-b844e13df510 None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:30:30.716926 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-c89a6f20-e3a3-40a5-8d56-b844e13df510 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:30:30.797091 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=98,keystone:INSERT=1,keystone:DELETE=1 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:30:30.813756 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1292/2558] 10.222.0.23 () {58 vars in 957 bytes} [Thu May 19 12:30:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:30:30.823026 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185c81c4-56e2-4c8a-81cf-3eaa5462e59c None None] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:30:30.823529 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185c81c4-56e2-4c8a-81cf-3eaa5462e59c None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:30:30.824093 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185c81c4-56e2-4c8a-81cf-3eaa5462e59c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:30:30.859768 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.auth.core [None req-185c81c4-56e2-4c8a-81cf-3eaa5462e59c None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:30:30.862333 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-185c81c4-56e2-4c8a-81cf-3eaa5462e59c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:30:30.879605 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1267/2559] 10.222.0.23 () {58 vars in 957 bytes} [Thu May 19 12:30:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:30:30.884217 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89a6f20-e3a3-40a5-8d56-b844e13df510 None None] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:30:30.899660 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89a6f20-e3a3-40a5-8d56-b844e13df510 None None] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:30:30.906798 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2eebaf-951a-4fd6-8488-7dd6071d5f82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:30:30.907551 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2eebaf-951a-4fd6-8488-7dd6071d5f82 admin admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:30:30.908031 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2eebaf-951a-4fd6-8488-7dd6071d5f82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:30:30.908374 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2eebaf-951a-4fd6-8488-7dd6071d5f82 admin admin] PATH_INFO: `/v3/domains` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:30:30.909288 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2eebaf-951a-4fd6-8488-7dd6071d5f82 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:30:30.909662 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2eebaf-951a-4fd6-8488-7dd6071d5f82 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:30:30.911562 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2eebaf-951a-4fd6-8488-7dd6071d5f82 admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:30:30.934402 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1293/2560] 10.222.0.23 () {58 vars in 1152 bytes} [Thu May 19 12:30:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 12:30:30.940396 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185c81c4-56e2-4c8a-81cf-3eaa5462e59c None None] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:30:30.953155 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185c81c4-56e2-4c8a-81cf-3eaa5462e59c None None] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:30:30.958436 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79fba2db-8747-4a03-94ca-e1f7a59a6aa9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:30:30.960815 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fba2db-8747-4a03-94ca-e1f7a59a6aa9 admin admin] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:30:30.960815 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fba2db-8747-4a03-94ca-e1f7a59a6aa9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:30:30.960815 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fba2db-8747-4a03-94ca-e1f7a59a6aa9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:30:30.960815 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79fba2db-8747-4a03-94ca-e1f7a59a6aa9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:30:30.962488 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79fba2db-8747-4a03-94ca-e1f7a59a6aa9 admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:30:31.082334 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1268/2561] 10.222.0.23 () {60 vars in 1152 bytes} [Thu May 19 12:30:30 2022] POST /identity/v3/projects => generated 379 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 12:30:31.084263 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2eebaf-951a-4fd6-8488-7dd6071d5f82 admin admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:30:31.097235 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2eebaf-951a-4fd6-8488-7dd6071d5f82 admin admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:30:31.195556 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41138749-48d2-4815-8e74-bcaf01dfea0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:30:31.196238 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41138749-48d2-4815-8e74-bcaf01dfea0c admin admin] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:30:31.196552 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41138749-48d2-4815-8e74-bcaf01dfea0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:30:31.196834 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41138749-48d2-4815-8e74-bcaf01dfea0c admin admin] PATH_INFO: `/v3/users` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:30:31.197583 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41138749-48d2-4815-8e74-bcaf01dfea0c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:30:31.200053 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41138749-48d2-4815-8e74-bcaf01dfea0c admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:30:31.318790 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1294/2562] 10.222.0.23 () {60 vars in 1143 bytes} [Thu May 19 12:30:31 2022] POST /identity/v3/users => generated 290 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 12:30:31.323665 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79fba2db-8747-4a03-94ca-e1f7a59a6aa9 admin admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:30:31.344986 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79fba2db-8747-4a03-94ca-e1f7a59a6aa9 admin admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:30:31.355836 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7a260c-1707-451a-85c9-a7c5e4d94908 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:30:31.358879 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7a260c-1707-451a-85c9-a7c5e4d94908 admin admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:30:31.358879 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7a260c-1707-451a-85c9-a7c5e4d94908 admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:30:31.358879 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7a260c-1707-451a-85c9-a7c5e4d94908 admin admin] PATH_INFO: `/v3/roles` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:30:31.358879 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7a260c-1707-451a-85c9-a7c5e4d94908 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:30:31.360719 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7a260c-1707-451a-85c9-a7c5e4d94908 admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:30:31.377282 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1269/2563] 10.222.0.23 () {58 vars in 1121 bytes} [Thu May 19 12:30:31 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 12:30:31.381671 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41138749-48d2-4815-8e74-bcaf01dfea0c admin admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:30:31.397812 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41138749-48d2-4815-8e74-bcaf01dfea0c admin admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:30:31.408179 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec9a173-acaf-413b-8e58-83ab003df88a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:30:31.408991 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec9a173-acaf-413b-8e58-83ab003df88a admin admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:30:31.409350 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec9a173-acaf-413b-8e58-83ab003df88a admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:30:31.409705 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec9a173-acaf-413b-8e58-83ab003df88a admin admin] PATH_INFO: `/v3/roles` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:30:31.410560 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec9a173-acaf-413b-8e58-83ab003df88a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:30:31.412895 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec9a173-acaf-413b-8e58-83ab003df88a admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:30:31.427229 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1295/2564] 10.222.0.23 () {58 vars in 1121 bytes} [Thu May 19 12:30:31 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 19 12:30:31.431955 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7a260c-1707-451a-85c9-a7c5e4d94908 admin admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:30:31.444856 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7a260c-1707-451a-85c9-a7c5e4d94908 admin admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:30:31.450452 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cce52f-1076-451d-b1fd-3c29bf0dc051 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:30:31.451294 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cce52f-1076-451d-b1fd-3c29bf0dc051 admin admin] REQUEST_METHOD: `PUT` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:30:31.451574 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cce52f-1076-451d-b1fd-3c29bf0dc051 admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:30:31.451904 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cce52f-1076-451d-b1fd-3c29bf0dc051 admin admin] PATH_INFO: `/v3/projects/f2c187bb046343368001964cd52917fb/users/4aa7ded4d4ce4f3086a36a4b3395aa6a/roles/259eb400cdb4439db72d14456ae9cd08` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:30:31.480855 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96cce52f-1076-451d-b1fd-3c29bf0dc051 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f2c187bb046343368001964cd52917fb, user_id=4aa7ded4d4ce4f3086a36a4b3395aa6a, role_id=259eb400cdb4439db72d14456ae9cd08)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:30:31.484701 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96cce52f-1076-451d-b1fd-3c29bf0dc051 admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:30:31.581116 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=79,keystone:DELETE=17,keystone:INSERT=4 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:30:31.587768 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1270/2565] 10.222.0.23 () {60 vars in 1482 bytes} [Thu May 19 12:30:31 2022] PUT /identity/v3/projects/f2c187bb046343368001964cd52917fb/users/4aa7ded4d4ce4f3086a36a4b3395aa6a/roles/259eb400cdb4439db72d14456ae9cd08 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:30:31.606387 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4363c991-4a1b-4eb5-aa37-5edc8d338c42 None None] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:30:31.606930 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4363c991-4a1b-4eb5-aa37-5edc8d338c42 None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:30:31.607226 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4363c991-4a1b-4eb5-aa37-5edc8d338c42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:30:31.639520 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.auth.core [None req-4363c991-4a1b-4eb5-aa37-5edc8d338c42 None None] MFA Rules not processed for user `4aa7ded4d4ce4f3086a36a4b3395aa6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:30:31.640838 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-4363c991-4a1b-4eb5-aa37-5edc8d338c42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:30:31.741709 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1296/2566] 10.222.0.23 () {58 vars in 957 bytes} [Thu May 19 12:30:31 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 12:30:31.750138 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cce52f-1076-451d-b1fd-3c29bf0dc051 admin admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:30:31.858684 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cce52f-1076-451d-b1fd-3c29bf0dc051 admin admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:30:31.874510 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d134923-1253-4989-91d5-f1b6a43e6977 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:30:31.875823 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d134923-1253-4989-91d5-f1b6a43e6977 admin admin] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:30:31.876627 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d134923-1253-4989-91d5-f1b6a43e6977 admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:30:31.877064 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d134923-1253-4989-91d5-f1b6a43e6977 admin admin] PATH_INFO: `/v3/projects` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:30:31.878136 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d134923-1253-4989-91d5-f1b6a43e6977 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:30:31.882322 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d134923-1253-4989-91d5-f1b6a43e6977 admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:30:31.998332 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1271/2567] 10.222.0.23 () {60 vars in 1152 bytes} [Thu May 19 12:30:31 2022] POST /identity/v3/projects => generated 379 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 12:30:32.005029 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4363c991-4a1b-4eb5-aa37-5edc8d338c42 None None] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:30:32.018940 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4363c991-4a1b-4eb5-aa37-5edc8d338c42 None None] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:30:32.108386 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:30:32.146002 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5653cdea-a2ad-4b10-9951-b38bd6c58c50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:30:32.147517 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5653cdea-a2ad-4b10-9951-b38bd6c58c50 admin admin] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:30:32.148128 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5653cdea-a2ad-4b10-9951-b38bd6c58c50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:30:32.148638 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5653cdea-a2ad-4b10-9951-b38bd6c58c50 admin admin] PATH_INFO: `/v3/users` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:30:32.152374 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5653cdea-a2ad-4b10-9951-b38bd6c58c50 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:30:32.157298 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5653cdea-a2ad-4b10-9951-b38bd6c58c50 admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:30:32.285290 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1297/2568] 10.222.0.23 () {60 vars in 1143 bytes} [Thu May 19 12:30:32 2022] POST /identity/v3/users => generated 296 bytes in 278 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 12:30:32.287168 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d134923-1253-4989-91d5-f1b6a43e6977 admin admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:30:32.302407 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d134923-1253-4989-91d5-f1b6a43e6977 admin admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:30:32.311694 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb8368b-12ea-49fe-abaf-d61ef9a0cdb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:30:32.312446 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb8368b-12ea-49fe-abaf-d61ef9a0cdb4 admin admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:30:32.312975 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb8368b-12ea-49fe-abaf-d61ef9a0cdb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:30:32.314391 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb8368b-12ea-49fe-abaf-d61ef9a0cdb4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:30:32.315212 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb8368b-12ea-49fe-abaf-d61ef9a0cdb4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:30:32.317072 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb8368b-12ea-49fe-abaf-d61ef9a0cdb4 admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:30:32.330338 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1272/2569] 10.222.0.23 () {58 vars in 1121 bytes} [Thu May 19 12:30:32 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 19 12:30:32.335964 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5653cdea-a2ad-4b10-9951-b38bd6c58c50 admin admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:30:32.355733 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5653cdea-a2ad-4b10-9951-b38bd6c58c50 admin admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:30:32.375478 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4137e767-c1b9-4b8b-8984-60f673ec836b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:30:32.378536 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4137e767-c1b9-4b8b-8984-60f673ec836b admin admin] REQUEST_METHOD: `PUT` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:30:32.378924 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4137e767-c1b9-4b8b-8984-60f673ec836b admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:30:32.379276 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4137e767-c1b9-4b8b-8984-60f673ec836b admin admin] PATH_INFO: `/v3/projects/c47f406c06934ab3a4a50f4220b7cba0/users/b795f6d9b998412b984ee499b8511274/roles/9db5671c65d3414ea2682afa2ccf1403` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:30:32.406987 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4137e767-c1b9-4b8b-8984-60f673ec836b admin admin] RBAC: Authorizing `identity:create_grant(project_id=c47f406c06934ab3a4a50f4220b7cba0, user_id=b795f6d9b998412b984ee499b8511274, role_id=9db5671c65d3414ea2682afa2ccf1403)` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:30:32.410542 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4137e767-c1b9-4b8b-8984-60f673ec836b admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:30:32.533297 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1298/2570] 10.222.0.23 () {60 vars in 1482 bytes} [Thu May 19 12:30:32 2022] PUT /identity/v3/projects/c47f406c06934ab3a4a50f4220b7cba0/users/b795f6d9b998412b984ee499b8511274/roles/9db5671c65d3414ea2682afa2ccf1403 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:30:32.538838 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb8368b-12ea-49fe-abaf-d61ef9a0cdb4 admin admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:30:32.556485 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb8368b-12ea-49fe-abaf-d61ef9a0cdb4 admin admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:30:32.671957 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7d52d8-26cc-4545-8506-de4f34529843 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:30:32.673401 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7d52d8-26cc-4545-8506-de4f34529843 admin admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:30:32.673903 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7d52d8-26cc-4545-8506-de4f34529843 admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:30:32.674390 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7d52d8-26cc-4545-8506-de4f34529843 admin admin] PATH_INFO: `/v3/roles` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:30:32.675871 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7d52d8-26cc-4545-8506-de4f34529843 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:30:32.678815 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7d52d8-26cc-4545-8506-de4f34529843 admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:30:32.702616 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1273/2571] 10.222.0.23 () {58 vars in 1121 bytes} [Thu May 19 12:30:32 2022] GET /identity/v3/roles => generated 1420 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 12:30:32.705493 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4137e767-c1b9-4b8b-8984-60f673ec836b admin admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:30:32.726032 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4137e767-c1b9-4b8b-8984-60f673ec836b admin admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:30:32.732412 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5ddd89-ecf4-483e-ae62-3fbc2b6dd2ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:30:32.733708 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5ddd89-ecf4-483e-ae62-3fbc2b6dd2ba admin admin] REQUEST_METHOD: `PUT` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:30:32.734005 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5ddd89-ecf4-483e-ae62-3fbc2b6dd2ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:30:32.734293 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5ddd89-ecf4-483e-ae62-3fbc2b6dd2ba admin admin] PATH_INFO: `/v3/projects/c47f406c06934ab3a4a50f4220b7cba0/users/b795f6d9b998412b984ee499b8511274/roles/9db5671c65d3414ea2682afa2ccf1403` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:30:32.738478 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5ddd89-ecf4-483e-ae62-3fbc2b6dd2ba admin admin] RBAC: Authorizing `identity:create_grant(project_id=c47f406c06934ab3a4a50f4220b7cba0, user_id=b795f6d9b998412b984ee499b8511274, role_id=9db5671c65d3414ea2682afa2ccf1403)` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:30:32.741326 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5ddd89-ecf4-483e-ae62-3fbc2b6dd2ba admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:30:32.763849 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1299/2572] 10.222.0.23 () {60 vars in 1482 bytes} [Thu May 19 12:30:32 2022] PUT /identity/v3/projects/c47f406c06934ab3a4a50f4220b7cba0/users/b795f6d9b998412b984ee499b8511274/roles/9db5671c65d3414ea2682afa2ccf1403 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:30:32.769821 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff78b19-e2e9-4fb2-ad2e-6d6a64e5aa2a None None] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:30:32.770226 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff78b19-e2e9-4fb2-ad2e-6d6a64e5aa2a None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:30:32.770587 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff78b19-e2e9-4fb2-ad2e-6d6a64e5aa2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:30:32.812744 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.auth.core [None req-aff78b19-e2e9-4fb2-ad2e-6d6a64e5aa2a None None] MFA Rules not processed for user `b795f6d9b998412b984ee499b8511274`. Rule list: `[]` (Enabled: `True`). {{(pid=80539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:30:32.814196 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-aff78b19-e2e9-4fb2-ad2e-6d6a64e5aa2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:30:32.899960 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:30:32.916482 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1274/2573] 10.222.0.23 () {58 vars in 957 bytes} [Thu May 19 12:30:32 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:30:32.931702 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5ddd89-ecf4-483e-ae62-3fbc2b6dd2ba admin admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:30:32.934578 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-ba5ddd89-ecf4-483e-ae62-3fbc2b6dd2ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:30:33.018506 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5ddd89-ecf4-483e-ae62-3fbc2b6dd2ba admin admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:30:33.023557 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd9fda9-97cf-4f9c-8cb6-57259c265872 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8800254c0e784e42a791fb9abc274d79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd70d25bf69314820b8aaecb65576978a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:30:33.024287 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd9fda9-97cf-4f9c-8cb6-57259c265872 service cinder] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:30:33.024577 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd9fda9-97cf-4f9c-8cb6-57259c265872 service cinder] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:30:33.024861 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd9fda9-97cf-4f9c-8cb6-57259c265872 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:30:33.047581 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd9fda9-97cf-4f9c-8cb6-57259c265872 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:30:33.050321 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd9fda9-97cf-4f9c-8cb6-57259c265872 service cinder] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:30:33.067257 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1300/2574] 10.222.0.23 () {60 vars in 1365 bytes} [Thu May 19 12:30:32 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:30:33.880028 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff78b19-e2e9-4fb2-ad2e-6d6a64e5aa2a None None] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:30:34.045241 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff78b19-e2e9-4fb2-ad2e-6d6a64e5aa2a None None] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:30:34.054210 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793fb7e2-05a9-423e-a620-127645eea9cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:30:34.059151 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793fb7e2-05a9-423e-a620-127645eea9cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:30:34.061039 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793fb7e2-05a9-423e-a620-127645eea9cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:30:34.062075 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793fb7e2-05a9-423e-a620-127645eea9cf admin admin] PATH_INFO: `/v3/users/4aa7ded4d4ce4f3086a36a4b3395aa6a` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:30:34.066780 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-793fb7e2-05a9-423e-a620-127645eea9cf admin admin] RBAC: Authorizing `identity:delete_user(user_id=4aa7ded4d4ce4f3086a36a4b3395aa6a)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:30:34.072471 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-793fb7e2-05a9-423e-a620-127645eea9cf admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:30:34.361730 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-793fb7e2-05a9-423e-a620-127645eea9cf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4aa7ded4d4ce4f3086a36a4b3395aa6a'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:30:34.389749 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-793fb7e2-05a9-423e-a620-127645eea9cf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4aa7ded4d4ce4f3086a36a4b3395aa6a'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:30:34.390828 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-793fb7e2-05a9-423e-a620-127645eea9cf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4aa7ded4d4ce4f3086a36a4b3395aa6a'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:30:34.500483 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-793fb7e2-05a9-423e-a620-127645eea9cf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4aa7ded4d4ce4f3086a36a4b3395aa6a'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:30:34.526862 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-793fb7e2-05a9-423e-a620-127645eea9cf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4aa7ded4d4ce4f3086a36a4b3395aa6a'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:30:34.543839 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1275/2575] 10.222.0.23 () {58 vars in 1223 bytes} [Thu May 19 12:30:33 2022] DELETE /identity/v3/users/4aa7ded4d4ce4f3086a36a4b3395aa6a => generated 0 bytes in 666 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:30:34.548447 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ca9925-5b68-484f-882d-f8c644094472 None None] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:30:34.551679 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ca9925-5b68-484f-882d-f8c644094472 None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:30:34.552132 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ca9925-5b68-484f-882d-f8c644094472 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:30:34.579897 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.auth.core [None req-07ca9925-5b68-484f-882d-f8c644094472 None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:30:34.582651 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-07ca9925-5b68-484f-882d-f8c644094472 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:30:34.724433 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1301/2576] 10.222.0.23 () {58 vars in 957 bytes} [Thu May 19 12:30:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 12:30:34.732597 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793fb7e2-05a9-423e-a620-127645eea9cf admin admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:30:34.734077 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-793fb7e2-05a9-423e-a620-127645eea9cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:30:34.830513 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793fb7e2-05a9-423e-a620-127645eea9cf admin admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:30:34.835235 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14cf0d81-0d9f-48f7-a2a7-74454ff4e9eb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a00b3fa7497456895e5e0bf105c4fc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd70d25bf69314820b8aaecb65576978a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:30:34.835956 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cf0d81-0d9f-48f7-a2a7-74454ff4e9eb service neutron] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:30:34.836442 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cf0d81-0d9f-48f7-a2a7-74454ff4e9eb service neutron] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:30:34.836861 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cf0d81-0d9f-48f7-a2a7-74454ff4e9eb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:30:34.848543 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14cf0d81-0d9f-48f7-a2a7-74454ff4e9eb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:30:34.854974 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14cf0d81-0d9f-48f7-a2a7-74454ff4e9eb service neutron] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:30:34.863917 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1276/2577] 10.222.0.23 () {60 vars in 1365 bytes} [Thu May 19 12:30:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:30:35.007749 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ca9925-5b68-484f-882d-f8c644094472 None None] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:30:35.009852 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-07ca9925-5b68-484f-882d-f8c644094472 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:30:35.030542 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ca9925-5b68-484f-882d-f8c644094472 None None] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:30:35.038864 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a92b3d-4d90-4ab6-9add-ab242767fe22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:30:35.040150 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a92b3d-4d90-4ab6-9add-ab242767fe22 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:30:35.040534 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a92b3d-4d90-4ab6-9add-ab242767fe22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:30:35.041050 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a92b3d-4d90-4ab6-9add-ab242767fe22 admin admin] PATH_INFO: `/v3/users/dd39ddcf4ab045b98a737ee529a2dcce` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:30:35.042941 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a92b3d-4d90-4ab6-9add-ab242767fe22 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dd39ddcf4ab045b98a737ee529a2dcce)` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:30:35.045511 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a92b3d-4d90-4ab6-9add-ab242767fe22 admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:30:35.308562 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=89,keystone:INSERT=5,keystone:DELETE=6 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:30:35.319761 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-06a92b3d-4d90-4ab6-9add-ab242767fe22 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dd39ddcf4ab045b98a737ee529a2dcce'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:30:35.341004 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-06a92b3d-4d90-4ab6-9add-ab242767fe22 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dd39ddcf4ab045b98a737ee529a2dcce'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:30:35.343889 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-06a92b3d-4d90-4ab6-9add-ab242767fe22 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dd39ddcf4ab045b98a737ee529a2dcce'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:30:35.444056 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-06a92b3d-4d90-4ab6-9add-ab242767fe22 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dd39ddcf4ab045b98a737ee529a2dcce'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:30:35.453741 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14cf0d81-0d9f-48f7-a2a7-74454ff4e9eb service neutron] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:30:35.456088 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-14cf0d81-0d9f-48f7-a2a7-74454ff4e9eb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:30:35.484401 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14cf0d81-0d9f-48f7-a2a7-74454ff4e9eb service neutron] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:30:35.490594 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-06a92b3d-4d90-4ab6-9add-ab242767fe22 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dd39ddcf4ab045b98a737ee529a2dcce'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:30:35.494858 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ef8599-bbaa-46c6-9548-067beed99509 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:30:35.496469 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ef8599-bbaa-46c6-9548-067beed99509 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:30:35.496981 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ef8599-bbaa-46c6-9548-067beed99509 admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:30:35.498482 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ef8599-bbaa-46c6-9548-067beed99509 admin admin] PATH_INFO: `/v3/projects/f2c187bb046343368001964cd52917fb` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:30:35.499749 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ef8599-bbaa-46c6-9548-067beed99509 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f2c187bb046343368001964cd52917fb)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:30:35.503234 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ef8599-bbaa-46c6-9548-067beed99509 admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:30:35.512394 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1302/2578] 10.222.0.23 () {58 vars in 1223 bytes} [Thu May 19 12:30:35 2022] DELETE /identity/v3/users/dd39ddcf4ab045b98a737ee529a2dcce => generated 0 bytes in 505 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:30:35.529636 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:30:35.720096 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-d7ef8599-bbaa-46c6-9548-067beed99509 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f2c187bb046343368001964cd52917fb', 'request_id': 'req-d7ef8599-bbaa-46c6-9548-067beed99509'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:30:35.733596 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-d7ef8599-bbaa-46c6-9548-067beed99509 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f2c187bb046343368001964cd52917fb', 'request_id': 'req-d7ef8599-bbaa-46c6-9548-067beed99509'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:30:35.849801 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-d7ef8599-bbaa-46c6-9548-067beed99509 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f2c187bb046343368001964cd52917fb', 'request_id': 'req-d7ef8599-bbaa-46c6-9548-067beed99509'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:30:35.868714 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-d7ef8599-bbaa-46c6-9548-067beed99509 admin admin] The token cache is being invalidate because project f2c187bb046343368001964cd52917fb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80539) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 12:30:35.869088 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-d7ef8599-bbaa-46c6-9548-067beed99509 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:30:35.870876 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-d7ef8599-bbaa-46c6-9548-067beed99509 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:30:35.876145 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1277/2579] 10.222.0.23 () {58 vars in 1232 bytes} [Thu May 19 12:30:35 2022] DELETE /identity/v3/projects/f2c187bb046343368001964cd52917fb => generated 0 bytes in 421 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 12:30:35.881183 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a92b3d-4d90-4ab6-9add-ab242767fe22 admin admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:30:35.882938 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-06a92b3d-4d90-4ab6-9add-ab242767fe22 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:30:35.896078 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ef8599-bbaa-46c6-9548-067beed99509 admin admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:30:35.898967 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-d7ef8599-bbaa-46c6-9548-067beed99509 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:30:36.036390 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ef8599-bbaa-46c6-9548-067beed99509 admin admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:30:36.044026 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8a4874-3f8d-4cc4-8ef5-4c9ab0514465 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:30:36.045210 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8a4874-3f8d-4cc4-8ef5-4c9ab0514465 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:30:36.045652 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8a4874-3f8d-4cc4-8ef5-4c9ab0514465 admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:30:36.046165 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8a4874-3f8d-4cc4-8ef5-4c9ab0514465 admin admin] PATH_INFO: `/v3/projects/1d2de86d602945238f5df0e2c388004f` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:30:36.047707 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8a4874-3f8d-4cc4-8ef5-4c9ab0514465 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1d2de86d602945238f5df0e2c388004f)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:30:36.050309 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8a4874-3f8d-4cc4-8ef5-4c9ab0514465 admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:30:36.066639 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a92b3d-4d90-4ab6-9add-ab242767fe22 admin admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:30:36.072165 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e1d4e6-bcfd-4f37-a4b6-6ff94c9bfceb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:30:36.073163 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e1d4e6-bcfd-4f37-a4b6-6ff94c9bfceb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:30:36.073461 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e1d4e6-bcfd-4f37-a4b6-6ff94c9bfceb admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:30:36.073736 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e1d4e6-bcfd-4f37-a4b6-6ff94c9bfceb admin admin] PATH_INFO: `/v3/users/b795f6d9b998412b984ee499b8511274` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:30:36.075525 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e1d4e6-bcfd-4f37-a4b6-6ff94c9bfceb admin admin] RBAC: Authorizing `identity:delete_user(user_id=b795f6d9b998412b984ee499b8511274)` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:30:36.077818 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e1d4e6-bcfd-4f37-a4b6-6ff94c9bfceb admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:30:36.196269 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-bd8a4874-3f8d-4cc4-8ef5-4c9ab0514465 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1d2de86d602945238f5df0e2c388004f', 'request_id': 'req-bd8a4874-3f8d-4cc4-8ef5-4c9ab0514465'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:30:36.204072 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-bd8a4874-3f8d-4cc4-8ef5-4c9ab0514465 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1d2de86d602945238f5df0e2c388004f', 'request_id': 'req-bd8a4874-3f8d-4cc4-8ef5-4c9ab0514465'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:30:36.312218 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-bd8a4874-3f8d-4cc4-8ef5-4c9ab0514465 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1d2de86d602945238f5df0e2c388004f', 'request_id': 'req-bd8a4874-3f8d-4cc4-8ef5-4c9ab0514465'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:30:36.321175 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-17e1d4e6-bcfd-4f37-a4b6-6ff94c9bfceb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b795f6d9b998412b984ee499b8511274'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:30:36.331213 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-bd8a4874-3f8d-4cc4-8ef5-4c9ab0514465 admin admin] The token cache is being invalidate because project 1d2de86d602945238f5df0e2c388004f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80539) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 12:30:36.331527 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-bd8a4874-3f8d-4cc4-8ef5-4c9ab0514465 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:30:36.332768 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-bd8a4874-3f8d-4cc4-8ef5-4c9ab0514465 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:30:36.336088 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1278/2580] 10.222.0.23 () {58 vars in 1232 bytes} [Thu May 19 12:30:35 2022] DELETE /identity/v3/projects/1d2de86d602945238f5df0e2c388004f => generated 0 bytes in 441 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 12:30:36.339296 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-17e1d4e6-bcfd-4f37-a4b6-6ff94c9bfceb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b795f6d9b998412b984ee499b8511274'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:30:36.339889 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-17e1d4e6-bcfd-4f37-a4b6-6ff94c9bfceb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b795f6d9b998412b984ee499b8511274'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:30:36.343302 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1116da5-7874-467f-8a2b-3d9eadef186b None None] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:30:36.344949 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1116da5-7874-467f-8a2b-3d9eadef186b None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:30:36.345738 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1116da5-7874-467f-8a2b-3d9eadef186b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:30:36.371199 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.auth.core [None req-d1116da5-7874-467f-8a2b-3d9eadef186b None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:30:36.373138 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-d1116da5-7874-467f-8a2b-3d9eadef186b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:30:36.411627 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:30:36.437435 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-17e1d4e6-bcfd-4f37-a4b6-6ff94c9bfceb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b795f6d9b998412b984ee499b8511274'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:30:36.450488 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-17e1d4e6-bcfd-4f37-a4b6-6ff94c9bfceb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b795f6d9b998412b984ee499b8511274'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:30:36.466832 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1279/2581] 10.222.0.23 () {58 vars in 957 bytes} [Thu May 19 12:30:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:30:36.467872 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1303/2582] 10.222.0.23 () {58 vars in 1223 bytes} [Thu May 19 12:30:35 2022] DELETE /identity/v3/users/b795f6d9b998412b984ee499b8511274 => generated 0 bytes in 587 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:30:36.481090 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7763fb4-702f-4a2d-93ab-ec98ae43319a None None] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:30:36.481482 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7763fb4-702f-4a2d-93ab-ec98ae43319a None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:30:36.481841 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7763fb4-702f-4a2d-93ab-ec98ae43319a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:30:36.505533 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.auth.core [None req-b7763fb4-702f-4a2d-93ab-ec98ae43319a None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:30:36.506796 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-b7763fb4-702f-4a2d-93ab-ec98ae43319a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:30:36.602117 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1280/2583] 10.222.0.23 () {58 vars in 957 bytes} [Thu May 19 12:30:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:30:36.607668 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e1d4e6-bcfd-4f37-a4b6-6ff94c9bfceb admin admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:30:36.622130 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e1d4e6-bcfd-4f37-a4b6-6ff94c9bfceb admin admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:30:36.627349 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f47068-1786-497d-b60e-570b3c0dfea1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:30:36.628115 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f47068-1786-497d-b60e-570b3c0dfea1 admin admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:30:36.628407 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f47068-1786-497d-b60e-570b3c0dfea1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:30:36.628691 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f47068-1786-497d-b60e-570b3c0dfea1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:30:36.629545 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f47068-1786-497d-b60e-570b3c0dfea1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:30:36.629884 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f47068-1786-497d-b60e-570b3c0dfea1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:30:36.631530 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f47068-1786-497d-b60e-570b3c0dfea1 admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:30:36.645445 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1304/2584] 10.222.0.23 () {58 vars in 1152 bytes} [Thu May 19 12:30:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:30:36.651768 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7763fb4-702f-4a2d-93ab-ec98ae43319a None None] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:30:36.663941 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7763fb4-702f-4a2d-93ab-ec98ae43319a None None] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:30:36.670177 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c49f349-f3d8-4208-a221-9eb83b1d790a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:30:36.670882 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c49f349-f3d8-4208-a221-9eb83b1d790a admin admin] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:30:36.671170 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c49f349-f3d8-4208-a221-9eb83b1d790a admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:30:36.671451 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c49f349-f3d8-4208-a221-9eb83b1d790a admin admin] PATH_INFO: `/v3/projects` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:30:36.672189 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c49f349-f3d8-4208-a221-9eb83b1d790a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:30:36.674126 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c49f349-f3d8-4208-a221-9eb83b1d790a admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:30:36.787934 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1281/2585] 10.222.0.23 () {60 vars in 1152 bytes} [Thu May 19 12:30:36 2022] POST /identity/v3/projects => generated 403 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 12:30:36.793543 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f47068-1786-497d-b60e-570b3c0dfea1 admin admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:30:36.811191 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f47068-1786-497d-b60e-570b3c0dfea1 admin admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:30:36.856110 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:30:36.931856 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8179413d-41a3-4a89-9504-2bf4e059850a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:30:36.933092 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8179413d-41a3-4a89-9504-2bf4e059850a admin admin] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:30:36.933359 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8179413d-41a3-4a89-9504-2bf4e059850a admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:30:36.933640 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8179413d-41a3-4a89-9504-2bf4e059850a admin admin] PATH_INFO: `/v3/users` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:30:36.934356 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8179413d-41a3-4a89-9504-2bf4e059850a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:30:36.936773 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8179413d-41a3-4a89-9504-2bf4e059850a admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:30:37.028403 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c49f349-f3d8-4208-a221-9eb83b1d790a admin admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:30:37.030578 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-9c49f349-f3d8-4208-a221-9eb83b1d790a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:30:37.036452 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1305/2586] 10.222.0.23 () {60 vars in 1143 bytes} [Thu May 19 12:30:36 2022] POST /identity/v3/users => generated 302 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 12:30:37.038466 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8179413d-41a3-4a89-9504-2bf4e059850a admin admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:30:37.050389 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c49f349-f3d8-4208-a221-9eb83b1d790a admin admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:30:37.058708 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8179413d-41a3-4a89-9504-2bf4e059850a admin admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:30:37.063811 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f500d5d-2307-48e2-977b-1dec89bfdb2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:30:37.068216 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f500d5d-2307-48e2-977b-1dec89bfdb2d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:30:37.068518 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f500d5d-2307-48e2-977b-1dec89bfdb2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:30:37.068651 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40068005-564f-4e37-ac1c-bb98fe834e24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:30:37.068827 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f500d5d-2307-48e2-977b-1dec89bfdb2d admin admin] PATH_INFO: `/v3/projects/c47f406c06934ab3a4a50f4220b7cba0` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:30:37.070415 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f500d5d-2307-48e2-977b-1dec89bfdb2d admin admin] RBAC: Authorizing `identity:delete_project(project_id=c47f406c06934ab3a4a50f4220b7cba0)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:30:37.071948 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40068005-564f-4e37-ac1c-bb98fe834e24 admin admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:30:37.073064 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f500d5d-2307-48e2-977b-1dec89bfdb2d admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:30:37.075013 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40068005-564f-4e37-ac1c-bb98fe834e24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:30:37.076008 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40068005-564f-4e37-ac1c-bb98fe834e24 admin admin] PATH_INFO: `/v3/roles` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:30:37.078346 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40068005-564f-4e37-ac1c-bb98fe834e24 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:30:37.084356 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40068005-564f-4e37-ac1c-bb98fe834e24 admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:30:37.110384 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1306/2587] 10.222.0.23 () {58 vars in 1121 bytes} [Thu May 19 12:30:37 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 12:30:37.115996 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40068005-564f-4e37-ac1c-bb98fe834e24 admin admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:30:37.137571 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40068005-564f-4e37-ac1c-bb98fe834e24 admin admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:30:37.145715 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584125d5-5f6f-4837-bf54-e2439f0e632b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:30:37.146912 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584125d5-5f6f-4837-bf54-e2439f0e632b admin admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:30:37.147451 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584125d5-5f6f-4837-bf54-e2439f0e632b admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:30:37.148003 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584125d5-5f6f-4837-bf54-e2439f0e632b admin admin] PATH_INFO: `/v3/roles` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:30:37.149802 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-584125d5-5f6f-4837-bf54-e2439f0e632b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:30:37.153886 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-584125d5-5f6f-4837-bf54-e2439f0e632b admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:30:37.174084 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1307/2588] 10.222.0.23 () {58 vars in 1121 bytes} [Thu May 19 12:30:37 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 12:30:37.179830 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584125d5-5f6f-4837-bf54-e2439f0e632b admin admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:30:37.194042 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584125d5-5f6f-4837-bf54-e2439f0e632b admin admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:30:37.214250 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-3f500d5d-2307-48e2-977b-1dec89bfdb2d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c47f406c06934ab3a4a50f4220b7cba0', 'request_id': 'req-3f500d5d-2307-48e2-977b-1dec89bfdb2d'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:30:37.225436 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-3f500d5d-2307-48e2-977b-1dec89bfdb2d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c47f406c06934ab3a4a50f4220b7cba0', 'request_id': 'req-3f500d5d-2307-48e2-977b-1dec89bfdb2d'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:30:37.279875 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8cf616-353e-425a-8b06-8eac4c86463f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:30:37.280884 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8cf616-353e-425a-8b06-8eac4c86463f admin admin] REQUEST_METHOD: `PUT` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:30:37.281176 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8cf616-353e-425a-8b06-8eac4c86463f admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:30:37.281454 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8cf616-353e-425a-8b06-8eac4c86463f admin admin] PATH_INFO: `/v3/projects/7dd1b178667e4d91ad125c3cbf030de6/users/e39605aae5544c9c8375d1ead49c0f37/roles/259eb400cdb4439db72d14456ae9cd08` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:30:37.299475 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8cf616-353e-425a-8b06-8eac4c86463f admin admin] RBAC: Authorizing `identity:create_grant(project_id=7dd1b178667e4d91ad125c3cbf030de6, user_id=e39605aae5544c9c8375d1ead49c0f37, role_id=259eb400cdb4439db72d14456ae9cd08)` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:30:37.302414 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8cf616-353e-425a-8b06-8eac4c86463f admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:30:37.328996 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-3f500d5d-2307-48e2-977b-1dec89bfdb2d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c47f406c06934ab3a4a50f4220b7cba0', 'request_id': 'req-3f500d5d-2307-48e2-977b-1dec89bfdb2d'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:30:37.345197 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-3f500d5d-2307-48e2-977b-1dec89bfdb2d admin admin] The token cache is being invalidate because project c47f406c06934ab3a4a50f4220b7cba0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80539) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 12:30:37.345657 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-3f500d5d-2307-48e2-977b-1dec89bfdb2d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:30:37.346495 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-3f500d5d-2307-48e2-977b-1dec89bfdb2d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:30:37.348537 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1282/2589] 10.222.0.23 () {58 vars in 1232 bytes} [Thu May 19 12:30:37 2022] DELETE /identity/v3/projects/c47f406c06934ab3a4a50f4220b7cba0 => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 12:30:37.355339 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63dbd8a-47a6-4f75-b3bd-ad3de86b9b86 None None] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:30:37.356294 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63dbd8a-47a6-4f75-b3bd-ad3de86b9b86 None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:30:37.356533 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63dbd8a-47a6-4f75-b3bd-ad3de86b9b86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:30:37.385045 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.auth.core [None req-b63dbd8a-47a6-4f75-b3bd-ad3de86b9b86 None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:30:37.385045 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-b63dbd8a-47a6-4f75-b3bd-ad3de86b9b86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:30:37.392036 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1283/2590] 10.222.0.23 () {58 vars in 957 bytes} [Thu May 19 12:30:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 12:30:37.400407 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c661477f-f371-48a5-97a1-b78c8da62e55 None None] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:30:37.401326 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c661477f-f371-48a5-97a1-b78c8da62e55 None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:30:37.401757 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c661477f-f371-48a5-97a1-b78c8da62e55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:30:37.423078 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.auth.core [None req-c661477f-f371-48a5-97a1-b78c8da62e55 None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:30:37.424979 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-c661477f-f371-48a5-97a1-b78c8da62e55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:30:37.433851 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1284/2591] 10.222.0.23 () {58 vars in 957 bytes} [Thu May 19 12:30:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 12:30:37.439738 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c661477f-f371-48a5-97a1-b78c8da62e55 None None] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:30:37.454757 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1308/2592] 10.222.0.23 () {60 vars in 1482 bytes} [Thu May 19 12:30:37 2022] PUT /identity/v3/projects/7dd1b178667e4d91ad125c3cbf030de6/users/e39605aae5544c9c8375d1ead49c0f37/roles/259eb400cdb4439db72d14456ae9cd08 => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 12:30:37.458777 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c661477f-f371-48a5-97a1-b78c8da62e55 None None] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:30:37.466604 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c14d84-9f32-4494-b5e2-48b8245ebc44 None None] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:30:37.467213 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c14d84-9f32-4494-b5e2-48b8245ebc44 None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:30:37.467608 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c14d84-9f32-4494-b5e2-48b8245ebc44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:30:37.474278 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:30:37.501575 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.auth.core [None req-60c14d84-9f32-4494-b5e2-48b8245ebc44 None None] MFA Rules not processed for user `e39605aae5544c9c8375d1ead49c0f37`. Rule list: `[]` (Enabled: `True`). {{(pid=80540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:30:37.503002 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-60c14d84-9f32-4494-b5e2-48b8245ebc44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:30:37.544102 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40dae952-a0a6-43d5-894b-40ef3a2f5bda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:30:37.546312 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40dae952-a0a6-43d5-894b-40ef3a2f5bda admin admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:30:37.548405 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40dae952-a0a6-43d5-894b-40ef3a2f5bda admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:30:37.548772 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40dae952-a0a6-43d5-894b-40ef3a2f5bda admin admin] PATH_INFO: `/v3/domains` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:30:37.550721 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40dae952-a0a6-43d5-894b-40ef3a2f5bda admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:30:37.551877 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40dae952-a0a6-43d5-894b-40ef3a2f5bda admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:30:37.557741 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40dae952-a0a6-43d5-894b-40ef3a2f5bda admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:30:37.587063 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1285/2593] 10.222.0.23 () {58 vars in 1152 bytes} [Thu May 19 12:30:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:30:37.590060 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40dae952-a0a6-43d5-894b-40ef3a2f5bda admin admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:30:37.602245 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1309/2594] 10.222.0.23 () {58 vars in 957 bytes} [Thu May 19 12:30:37 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:30:37.606669 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40dae952-a0a6-43d5-894b-40ef3a2f5bda admin admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:30:37.608295 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c14d84-9f32-4494-b5e2-48b8245ebc44 None None] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:30:37.610287 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-60c14d84-9f32-4494-b5e2-48b8245ebc44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:30:37.613079 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46bfcc3d-17f7-4fbd-b988-9a5d5bda5101 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:30:37.613893 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bfcc3d-17f7-4fbd-b988-9a5d5bda5101 admin admin] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:30:37.614280 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bfcc3d-17f7-4fbd-b988-9a5d5bda5101 admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:30:37.614642 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bfcc3d-17f7-4fbd-b988-9a5d5bda5101 admin admin] PATH_INFO: `/v3/projects` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:30:37.615907 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46bfcc3d-17f7-4fbd-b988-9a5d5bda5101 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:30:37.618094 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46bfcc3d-17f7-4fbd-b988-9a5d5bda5101 admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:30:37.628385 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:30:37.635730 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c14d84-9f32-4494-b5e2-48b8245ebc44 None None] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:30:37.643465 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df0fcb2-d35b-4317-b0e6-e1b3d68d1558 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:30:37.644613 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df0fcb2-d35b-4317-b0e6-e1b3d68d1558 admin admin] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:30:37.645049 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df0fcb2-d35b-4317-b0e6-e1b3d68d1558 admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:30:37.645538 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df0fcb2-d35b-4317-b0e6-e1b3d68d1558 admin admin] PATH_INFO: `/v3/projects` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:30:37.646733 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df0fcb2-d35b-4317-b0e6-e1b3d68d1558 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:30:37.650280 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df0fcb2-d35b-4317-b0e6-e1b3d68d1558 admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:30:37.738700 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1286/2595] 10.222.0.23 () {60 vars in 1152 bytes} [Thu May 19 12:30:37 2022] POST /identity/v3/projects => generated 383 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 12:30:37.743066 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46bfcc3d-17f7-4fbd-b988-9a5d5bda5101 admin admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:30:37.785348 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1310/2596] 10.222.0.23 () {60 vars in 1152 bytes} [Thu May 19 12:30:37 2022] POST /identity/v3/projects => generated 403 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 12:30:37.787314 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df0fcb2-d35b-4317-b0e6-e1b3d68d1558 admin admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:30:37.850481 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46bfcc3d-17f7-4fbd-b988-9a5d5bda5101 admin admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:30:37.911174 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df0fcb2-d35b-4317-b0e6-e1b3d68d1558 admin admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:30:37.917630 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51630450-c777-484a-a617-671aa991ade1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:30:37.918339 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51630450-c777-484a-a617-671aa991ade1 admin admin] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:30:37.918643 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51630450-c777-484a-a617-671aa991ade1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:30:37.918914 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51630450-c777-484a-a617-671aa991ade1 admin admin] PATH_INFO: `/v3/users` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:30:37.919643 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51630450-c777-484a-a617-671aa991ade1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:30:37.921985 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51630450-c777-484a-a617-671aa991ade1 admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:30:37.957016 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c07eb62-8c34-45d3-b012-2461b303ea95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:30:37.958113 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c07eb62-8c34-45d3-b012-2461b303ea95 admin admin] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:30:37.958592 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c07eb62-8c34-45d3-b012-2461b303ea95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:30:37.958979 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c07eb62-8c34-45d3-b012-2461b303ea95 admin admin] PATH_INFO: `/v3/users` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:30:37.959775 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c07eb62-8c34-45d3-b012-2461b303ea95 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:30:37.962344 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c07eb62-8c34-45d3-b012-2461b303ea95 admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:30:38.057473 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1311/2597] 10.222.0.23 () {60 vars in 1143 bytes} [Thu May 19 12:30:37 2022] POST /identity/v3/users => generated 308 bytes in 271 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 12:30:38.062052 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51630450-c777-484a-a617-671aa991ade1 admin admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:30:38.073569 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51630450-c777-484a-a617-671aa991ade1 admin admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:30:38.080284 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4177eb74-f395-4511-8dbc-1a2fee514f95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:30:38.081057 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4177eb74-f395-4511-8dbc-1a2fee514f95 admin admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:30:38.081341 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4177eb74-f395-4511-8dbc-1a2fee514f95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:30:38.081624 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4177eb74-f395-4511-8dbc-1a2fee514f95 admin admin] PATH_INFO: `/v3/roles` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:30:38.082379 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4177eb74-f395-4511-8dbc-1a2fee514f95 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:30:38.084092 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4177eb74-f395-4511-8dbc-1a2fee514f95 admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:30:38.096529 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1312/2598] 10.222.0.23 () {58 vars in 1121 bytes} [Thu May 19 12:30:38 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 19 12:30:38.104533 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4177eb74-f395-4511-8dbc-1a2fee514f95 admin admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:30:38.115538 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1287/2599] 10.222.0.23 () {60 vars in 1143 bytes} [Thu May 19 12:30:37 2022] POST /identity/v3/users => generated 292 bytes in 370 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 12:30:38.117449 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c07eb62-8c34-45d3-b012-2461b303ea95 admin admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:30:38.119076 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4177eb74-f395-4511-8dbc-1a2fee514f95 admin admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:30:38.127856 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ed8331-de72-4a29-a8cd-1a658bc3edcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:30:38.128998 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ed8331-de72-4a29-a8cd-1a658bc3edcb admin admin] REQUEST_METHOD: `PUT` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:30:38.129879 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ed8331-de72-4a29-a8cd-1a658bc3edcb admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:30:38.130287 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ed8331-de72-4a29-a8cd-1a658bc3edcb admin admin] PATH_INFO: `/v3/projects/0c5478cf18984859ad9fd5f501afe0da/users/ce34e262e9c84dfaa58d50b2e643daec/roles/9db5671c65d3414ea2682afa2ccf1403` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:30:38.133493 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c07eb62-8c34-45d3-b012-2461b303ea95 admin admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:30:38.139982 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34f1e3c-19a6-40f2-bad1-543dde539f93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:30:38.140687 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34f1e3c-19a6-40f2-bad1-543dde539f93 admin admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:30:38.141185 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34f1e3c-19a6-40f2-bad1-543dde539f93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:30:38.141359 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34f1e3c-19a6-40f2-bad1-543dde539f93 admin admin] PATH_INFO: `/v3/roles` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:30:38.142107 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34f1e3c-19a6-40f2-bad1-543dde539f93 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:30:38.144162 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34f1e3c-19a6-40f2-bad1-543dde539f93 admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:30:38.159956 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1288/2600] 10.222.0.23 () {58 vars in 1121 bytes} [Thu May 19 12:30:38 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 19 12:30:38.160237 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ed8331-de72-4a29-a8cd-1a658bc3edcb admin admin] RBAC: Authorizing `identity:create_grant(project_id=0c5478cf18984859ad9fd5f501afe0da, user_id=ce34e262e9c84dfaa58d50b2e643daec, role_id=9db5671c65d3414ea2682afa2ccf1403)` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:30:38.167990 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ed8331-de72-4a29-a8cd-1a658bc3edcb admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:30:38.168197 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34f1e3c-19a6-40f2-bad1-543dde539f93 admin admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:30:38.180642 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34f1e3c-19a6-40f2-bad1-543dde539f93 admin admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:30:38.188510 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa3f8ed-efce-46f0-8f2c-df3eeb186b06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:30:38.189398 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa3f8ed-efce-46f0-8f2c-df3eeb186b06 admin admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:30:38.189872 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa3f8ed-efce-46f0-8f2c-df3eeb186b06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:30:38.190211 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa3f8ed-efce-46f0-8f2c-df3eeb186b06 admin admin] PATH_INFO: `/v3/roles` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:30:38.191327 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa3f8ed-efce-46f0-8f2c-df3eeb186b06 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:30:38.193398 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa3f8ed-efce-46f0-8f2c-df3eeb186b06 admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:30:38.209229 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1289/2601] 10.222.0.23 () {58 vars in 1121 bytes} [Thu May 19 12:30:38 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 19 12:30:38.214908 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa3f8ed-efce-46f0-8f2c-df3eeb186b06 admin admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:30:38.222468 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:30:38.226794 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa3f8ed-efce-46f0-8f2c-df3eeb186b06 admin admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:30:38.235744 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20c7103-6610-4583-8b0f-744f540c9a80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:30:38.237687 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20c7103-6610-4583-8b0f-744f540c9a80 admin admin] REQUEST_METHOD: `PUT` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:30:38.238148 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20c7103-6610-4583-8b0f-744f540c9a80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:30:38.238618 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20c7103-6610-4583-8b0f-744f540c9a80 admin admin] PATH_INFO: `/v3/projects/2408c708d3ab4f6a9778c9634765d187/users/dad53d7fec2849908153ce3def3c1145/roles/259eb400cdb4439db72d14456ae9cd08` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:30:38.260689 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20c7103-6610-4583-8b0f-744f540c9a80 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2408c708d3ab4f6a9778c9634765d187, user_id=dad53d7fec2849908153ce3def3c1145, role_id=259eb400cdb4439db72d14456ae9cd08)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:30:38.264281 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20c7103-6610-4583-8b0f-744f540c9a80 admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:30:38.290336 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1313/2602] 10.222.0.23 () {60 vars in 1482 bytes} [Thu May 19 12:30:38 2022] PUT /identity/v3/projects/0c5478cf18984859ad9fd5f501afe0da/users/ce34e262e9c84dfaa58d50b2e643daec/roles/9db5671c65d3414ea2682afa2ccf1403 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 12:30:38.296645 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ed8331-de72-4a29-a8cd-1a658bc3edcb admin admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:30:38.386847 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ed8331-de72-4a29-a8cd-1a658bc3edcb admin admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:30:38.396889 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1290/2603] 10.222.0.23 () {60 vars in 1482 bytes} [Thu May 19 12:30:38 2022] PUT /identity/v3/projects/2408c708d3ab4f6a9778c9634765d187/users/dad53d7fec2849908153ce3def3c1145/roles/259eb400cdb4439db72d14456ae9cd08 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 12:30:38.415229 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571457bd-467c-4b81-a0c6-119eef58ab97 None None] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:30:38.415562 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571457bd-467c-4b81-a0c6-119eef58ab97 None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:30:38.415939 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571457bd-467c-4b81-a0c6-119eef58ab97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:30:38.445455 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:30:38.464658 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.auth.core [None req-571457bd-467c-4b81-a0c6-119eef58ab97 None None] MFA Rules not processed for user `dad53d7fec2849908153ce3def3c1145`. Rule list: `[]` (Enabled: `True`). {{(pid=80539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:30:38.466132 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-571457bd-467c-4b81-a0c6-119eef58ab97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:30:38.467749 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e894cccd-6a7a-471b-838c-2d8120c19f3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:30:38.468501 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e894cccd-6a7a-471b-838c-2d8120c19f3b admin admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:30:38.468794 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e894cccd-6a7a-471b-838c-2d8120c19f3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:30:38.469077 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e894cccd-6a7a-471b-838c-2d8120c19f3b admin admin] PATH_INFO: `/v3/roles` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:30:38.469840 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e894cccd-6a7a-471b-838c-2d8120c19f3b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:30:38.471551 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e894cccd-6a7a-471b-838c-2d8120c19f3b admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:30:38.485325 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1314/2604] 10.222.0.23 () {58 vars in 1121 bytes} [Thu May 19 12:30:38 2022] GET /identity/v3/roles => generated 1420 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 19 12:30:38.491152 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e894cccd-6a7a-471b-838c-2d8120c19f3b admin admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:30:38.504241 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e894cccd-6a7a-471b-838c-2d8120c19f3b admin admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:30:38.518113 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f02c21d-76af-47b2-a254-09322daad4a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:30:38.519510 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f02c21d-76af-47b2-a254-09322daad4a5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:30:38.519838 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f02c21d-76af-47b2-a254-09322daad4a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:30:38.520114 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f02c21d-76af-47b2-a254-09322daad4a5 admin admin] PATH_INFO: `/v3/projects/0c5478cf18984859ad9fd5f501afe0da/users/ce34e262e9c84dfaa58d50b2e643daec/roles/9db5671c65d3414ea2682afa2ccf1403` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:30:38.522870 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f02c21d-76af-47b2-a254-09322daad4a5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0c5478cf18984859ad9fd5f501afe0da, user_id=ce34e262e9c84dfaa58d50b2e643daec, role_id=9db5671c65d3414ea2682afa2ccf1403)` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:30:38.528164 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f02c21d-76af-47b2-a254-09322daad4a5 admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:30:38.547899 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1315/2605] 10.222.0.23 () {60 vars in 1482 bytes} [Thu May 19 12:30:38 2022] PUT /identity/v3/projects/0c5478cf18984859ad9fd5f501afe0da/users/ce34e262e9c84dfaa58d50b2e643daec/roles/9db5671c65d3414ea2682afa2ccf1403 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:30:38.552852 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ece3bf-c907-482b-a3a7-b9e09ecae38e None None] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:30:38.553258 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ece3bf-c907-482b-a3a7-b9e09ecae38e None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:30:38.553723 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ece3bf-c907-482b-a3a7-b9e09ecae38e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:30:38.592101 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.auth.core [None req-d0ece3bf-c907-482b-a3a7-b9e09ecae38e None None] MFA Rules not processed for user `ce34e262e9c84dfaa58d50b2e643daec`. Rule list: `[]` (Enabled: `True`). {{(pid=80540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:30:38.593919 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-d0ece3bf-c907-482b-a3a7-b9e09ecae38e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:30:38.601814 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1291/2606] 10.222.0.23 () {58 vars in 957 bytes} [Thu May 19 12:30:38 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:30:38.612870 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571457bd-467c-4b81-a0c6-119eef58ab97 None None] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:30:38.615590 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-571457bd-467c-4b81-a0c6-119eef58ab97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:30:38.725809 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571457bd-467c-4b81-a0c6-119eef58ab97 None None] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:30:38.730057 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1316/2607] 10.222.0.23 () {58 vars in 957 bytes} [Thu May 19 12:30:38 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 12:30:38.748562 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ece3bf-c907-482b-a3a7-b9e09ecae38e None None] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:30:38.750981 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-d0ece3bf-c907-482b-a3a7-b9e09ecae38e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:30:38.813426 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5980f46d-cc05-4d5a-8850-72ed65c4c79a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8800254c0e784e42a791fb9abc274d79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd70d25bf69314820b8aaecb65576978a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:30:38.814344 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5980f46d-cc05-4d5a-8850-72ed65c4c79a service cinder] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:30:38.814885 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5980f46d-cc05-4d5a-8850-72ed65c4c79a service cinder] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:30:38.815150 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5980f46d-cc05-4d5a-8850-72ed65c4c79a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:30:38.869917 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ece3bf-c907-482b-a3a7-b9e09ecae38e None None] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:30:38.877016 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f45a9c-0015-485d-8e6a-b51531318478 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8800254c0e784e42a791fb9abc274d79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd70d25bf69314820b8aaecb65576978a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:30:38.878170 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f45a9c-0015-485d-8e6a-b51531318478 service cinder] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:30:38.878629 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f45a9c-0015-485d-8e6a-b51531318478 service cinder] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:30:38.879039 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f45a9c-0015-485d-8e6a-b51531318478 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:30:38.894748 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:30:38.899077 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f45a9c-0015-485d-8e6a-b51531318478 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:30:38.903221 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f45a9c-0015-485d-8e6a-b51531318478 service cinder] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:30:38.919397 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5980f46d-cc05-4d5a-8850-72ed65c4c79a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:30:38.921028 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5980f46d-cc05-4d5a-8850-72ed65c4c79a service cinder] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:30:38.924106 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1317/2608] 10.222.0.23 () {60 vars in 1365 bytes} [Thu May 19 12:30:38 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:30:38.933612 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1292/2609] 10.222.0.23 () {60 vars in 1365 bytes} [Thu May 19 12:30:38 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:30:38.960404 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f45a9c-0015-485d-8e6a-b51531318478 service cinder] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:30:39.051642 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:30:39.061167 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f45a9c-0015-485d-8e6a-b51531318478 service cinder] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:30:39.067229 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15024a9-2dee-4abf-8937-f9acc037aea3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:30:39.068536 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15024a9-2dee-4abf-8937-f9acc037aea3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:30:39.069005 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15024a9-2dee-4abf-8937-f9acc037aea3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:30:39.069414 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15024a9-2dee-4abf-8937-f9acc037aea3 admin admin] PATH_INFO: `/v3/users/dad53d7fec2849908153ce3def3c1145` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:30:39.071277 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15024a9-2dee-4abf-8937-f9acc037aea3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dad53d7fec2849908153ce3def3c1145)` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:30:39.074876 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15024a9-2dee-4abf-8937-f9acc037aea3 admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:30:39.163751 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5980f46d-cc05-4d5a-8850-72ed65c4c79a service cinder] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:30:39.180228 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5980f46d-cc05-4d5a-8850-72ed65c4c79a service cinder] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:30:39.188688 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d782d026-3fcd-467d-a35b-af6b192dcc4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:30:39.190402 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d782d026-3fcd-467d-a35b-af6b192dcc4e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:30:39.190772 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d782d026-3fcd-467d-a35b-af6b192dcc4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:30:39.191238 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d782d026-3fcd-467d-a35b-af6b192dcc4e admin admin] PATH_INFO: `/v3/users/e39605aae5544c9c8375d1ead49c0f37` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:30:39.193348 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d782d026-3fcd-467d-a35b-af6b192dcc4e admin admin] RBAC: Authorizing `identity:delete_user(user_id=e39605aae5544c9c8375d1ead49c0f37)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:30:39.196063 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d782d026-3fcd-467d-a35b-af6b192dcc4e admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:30:39.343015 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-d15024a9-2dee-4abf-8937-f9acc037aea3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dad53d7fec2849908153ce3def3c1145'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:30:39.364618 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-d15024a9-2dee-4abf-8937-f9acc037aea3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dad53d7fec2849908153ce3def3c1145'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:30:39.365641 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-d15024a9-2dee-4abf-8937-f9acc037aea3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dad53d7fec2849908153ce3def3c1145'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:30:39.468863 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-d782d026-3fcd-467d-a35b-af6b192dcc4e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e39605aae5544c9c8375d1ead49c0f37'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:30:39.487741 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-d782d026-3fcd-467d-a35b-af6b192dcc4e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e39605aae5544c9c8375d1ead49c0f37'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:30:39.488687 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-d782d026-3fcd-467d-a35b-af6b192dcc4e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e39605aae5544c9c8375d1ead49c0f37'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:30:39.506478 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-d15024a9-2dee-4abf-8937-f9acc037aea3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dad53d7fec2849908153ce3def3c1145'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:30:39.520418 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-d15024a9-2dee-4abf-8937-f9acc037aea3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dad53d7fec2849908153ce3def3c1145'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:30:39.530657 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1318/2610] 10.222.0.23 () {58 vars in 1223 bytes} [Thu May 19 12:30:38 2022] DELETE /identity/v3/users/dad53d7fec2849908153ce3def3c1145 => generated 0 bytes in 572 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:30:39.535762 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244254c7-559b-424f-bdef-8e8c57a38309 None None] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:30:39.536892 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244254c7-559b-424f-bdef-8e8c57a38309 None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:30:39.537248 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244254c7-559b-424f-bdef-8e8c57a38309 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:30:39.559673 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.auth.core [None req-244254c7-559b-424f-bdef-8e8c57a38309 None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:30:39.562251 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-244254c7-559b-424f-bdef-8e8c57a38309 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:30:39.614350 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-d782d026-3fcd-467d-a35b-af6b192dcc4e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e39605aae5544c9c8375d1ead49c0f37'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:30:39.627936 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-d782d026-3fcd-467d-a35b-af6b192dcc4e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e39605aae5544c9c8375d1ead49c0f37'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:30:39.637845 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1293/2611] 10.222.0.23 () {58 vars in 1223 bytes} [Thu May 19 12:30:39 2022] DELETE /identity/v3/users/e39605aae5544c9c8375d1ead49c0f37 => generated 0 bytes in 475 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 12:30:39.643311 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1319/2612] 10.222.0.23 () {58 vars in 957 bytes} [Thu May 19 12:30:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 12:30:39.643579 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3006d3aa-4ed1-4d31-8cc0-30e2cfb49529 None None] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:30:39.644103 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3006d3aa-4ed1-4d31-8cc0-30e2cfb49529 None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:30:39.644476 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3006d3aa-4ed1-4d31-8cc0-30e2cfb49529 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:30:39.662845 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244254c7-559b-424f-bdef-8e8c57a38309 None None] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:30:39.665247 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-244254c7-559b-424f-bdef-8e8c57a38309 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:30:39.673632 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.auth.core [None req-3006d3aa-4ed1-4d31-8cc0-30e2cfb49529 None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:30:39.677148 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-3006d3aa-4ed1-4d31-8cc0-30e2cfb49529 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:30:39.762007 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244254c7-559b-424f-bdef-8e8c57a38309 None None] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:30:39.796564 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1294/2613] 10.222.0.23 () {58 vars in 957 bytes} [Thu May 19 12:30:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 12:30:39.812871 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3006d3aa-4ed1-4d31-8cc0-30e2cfb49529 None None] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:30:39.814626 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-3006d3aa-4ed1-4d31-8cc0-30e2cfb49529 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:30:39.822327 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:30:39.851557 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22f8768-9bc5-4e6a-8c6e-16d80b61f179 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a00b3fa7497456895e5e0bf105c4fc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd70d25bf69314820b8aaecb65576978a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:30:39.852378 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22f8768-9bc5-4e6a-8c6e-16d80b61f179 service neutron] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:30:39.860822 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22f8768-9bc5-4e6a-8c6e-16d80b61f179 service neutron] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:30:39.860822 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22f8768-9bc5-4e6a-8c6e-16d80b61f179 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:30:39.866450 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22f8768-9bc5-4e6a-8c6e-16d80b61f179 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:30:39.868587 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22f8768-9bc5-4e6a-8c6e-16d80b61f179 service neutron] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:30:39.890668 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1320/2614] 10.222.0.23 () {60 vars in 1365 bytes} [Thu May 19 12:30:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 228 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 12:30:39.894383 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22f8768-9bc5-4e6a-8c6e-16d80b61f179 service neutron] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:30:39.894383 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-c22f8768-9bc5-4e6a-8c6e-16d80b61f179 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:30:39.949668 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3006d3aa-4ed1-4d31-8cc0-30e2cfb49529 None None] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:30:39.956132 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866683c3-e9c8-4955-9a99-cd7033f657dd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a00b3fa7497456895e5e0bf105c4fc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd70d25bf69314820b8aaecb65576978a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:30:39.957020 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866683c3-e9c8-4955-9a99-cd7033f657dd service neutron] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:30:39.957377 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866683c3-e9c8-4955-9a99-cd7033f657dd service neutron] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:30:39.963438 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866683c3-e9c8-4955-9a99-cd7033f657dd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:30:39.973183 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:30:39.981475 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866683c3-e9c8-4955-9a99-cd7033f657dd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:30:39.983499 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866683c3-e9c8-4955-9a99-cd7033f657dd service neutron] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:30:40.001169 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22f8768-9bc5-4e6a-8c6e-16d80b61f179 service neutron] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:30:40.002512 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1295/2615] 10.222.0.23 () {60 vars in 1365 bytes} [Thu May 19 12:30:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 12:30:40.012995 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79cb1116-200d-45a9-a0e7-bedbfb51d961 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8800254c0e784e42a791fb9abc274d79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd70d25bf69314820b8aaecb65576978a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:30:40.014739 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cb1116-200d-45a9-a0e7-bedbfb51d961 service cinder] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:30:40.015107 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cb1116-200d-45a9-a0e7-bedbfb51d961 service cinder] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:30:40.020082 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cb1116-200d-45a9-a0e7-bedbfb51d961 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:30:40.020082 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-79cb1116-200d-45a9-a0e7-bedbfb51d961 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:30:40.150892 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79cb1116-200d-45a9-a0e7-bedbfb51d961 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:30:40.152560 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79cb1116-200d-45a9-a0e7-bedbfb51d961 service cinder] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:30:40.177514 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1321/2616] 10.222.0.23 () {60 vars in 1365 bytes} [Thu May 19 12:30:39 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 286 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:30:40.747323 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866683c3-e9c8-4955-9a99-cd7033f657dd service neutron] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:30:40.748224 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79cb1116-200d-45a9-a0e7-bedbfb51d961 service cinder] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:30:40.750809 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-79cb1116-200d-45a9-a0e7-bedbfb51d961 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:30:40.751950 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-866683c3-e9c8-4955-9a99-cd7033f657dd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:30:40.790665 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866683c3-e9c8-4955-9a99-cd7033f657dd service neutron] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:30:40.801707 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79cb1116-200d-45a9-a0e7-bedbfb51d961 service cinder] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:30:40.810851 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4081454d-da40-4d2f-90e9-ac281c1a6ae0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:30:40.811510 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8605120a-72c4-4eeb-a39e-0c033ec2a6d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:30:40.812334 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4081454d-da40-4d2f-90e9-ac281c1a6ae0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:30:40.812799 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4081454d-da40-4d2f-90e9-ac281c1a6ae0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:30:40.813151 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8605120a-72c4-4eeb-a39e-0c033ec2a6d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:30:40.813318 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4081454d-da40-4d2f-90e9-ac281c1a6ae0 admin admin] PATH_INFO: `/v3/projects/7dd1b178667e4d91ad125c3cbf030de6` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:30:40.813608 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8605120a-72c4-4eeb-a39e-0c033ec2a6d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:30:40.814060 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8605120a-72c4-4eeb-a39e-0c033ec2a6d4 admin admin] PATH_INFO: `/v3/projects/2408c708d3ab4f6a9778c9634765d187` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:30:40.817492 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4081454d-da40-4d2f-90e9-ac281c1a6ae0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7dd1b178667e4d91ad125c3cbf030de6)` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:30:40.819413 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8605120a-72c4-4eeb-a39e-0c033ec2a6d4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2408c708d3ab4f6a9778c9634765d187)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:30:40.823366 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4081454d-da40-4d2f-90e9-ac281c1a6ae0 admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:30:40.829312 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8605120a-72c4-4eeb-a39e-0c033ec2a6d4 admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:30:41.095394 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-4081454d-da40-4d2f-90e9-ac281c1a6ae0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7dd1b178667e4d91ad125c3cbf030de6', 'request_id': 'req-4081454d-da40-4d2f-90e9-ac281c1a6ae0'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:30:41.100564 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-8605120a-72c4-4eeb-a39e-0c033ec2a6d4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2408c708d3ab4f6a9778c9634765d187', 'request_id': 'req-8605120a-72c4-4eeb-a39e-0c033ec2a6d4'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:30:41.109086 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-4081454d-da40-4d2f-90e9-ac281c1a6ae0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7dd1b178667e4d91ad125c3cbf030de6', 'request_id': 'req-4081454d-da40-4d2f-90e9-ac281c1a6ae0'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:30:41.111402 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:30:41.116458 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-8605120a-72c4-4eeb-a39e-0c033ec2a6d4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2408c708d3ab4f6a9778c9634765d187', 'request_id': 'req-8605120a-72c4-4eeb-a39e-0c033ec2a6d4'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:30:41.122414 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-4081454d-da40-4d2f-90e9-ac281c1a6ae0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7dd1b178667e4d91ad125c3cbf030de6', 'request_id': 'req-4081454d-da40-4d2f-90e9-ac281c1a6ae0'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:30:41.249958 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-8605120a-72c4-4eeb-a39e-0c033ec2a6d4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2408c708d3ab4f6a9778c9634765d187', 'request_id': 'req-8605120a-72c4-4eeb-a39e-0c033ec2a6d4'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:30:41.269810 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-8605120a-72c4-4eeb-a39e-0c033ec2a6d4 admin admin] The token cache is being invalidate because project 2408c708d3ab4f6a9778c9634765d187 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80539) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 12:30:41.270105 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-8605120a-72c4-4eeb-a39e-0c033ec2a6d4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:30:41.270803 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-8605120a-72c4-4eeb-a39e-0c033ec2a6d4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:30:41.272729 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1296/2617] 10.222.0.23 () {58 vars in 1232 bytes} [Thu May 19 12:30:40 2022] DELETE /identity/v3/projects/2408c708d3ab4f6a9778c9634765d187 => generated 0 bytes in 526 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:30:41.278430 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3570c9ae-2e49-4701-ac19-05f42c50d3dd None None] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:30:41.280060 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3570c9ae-2e49-4701-ac19-05f42c50d3dd None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:30:41.280060 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3570c9ae-2e49-4701-ac19-05f42c50d3dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:30:41.301935 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.auth.core [None req-3570c9ae-2e49-4701-ac19-05f42c50d3dd None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:30:41.303069 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-3570c9ae-2e49-4701-ac19-05f42c50d3dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:30:41.369876 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-4081454d-da40-4d2f-90e9-ac281c1a6ae0 admin admin] The token cache is being invalidate because project 7dd1b178667e4d91ad125c3cbf030de6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80540) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 12:30:41.370674 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-4081454d-da40-4d2f-90e9-ac281c1a6ae0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:30:41.371413 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-4081454d-da40-4d2f-90e9-ac281c1a6ae0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:30:41.375926 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1322/2618] 10.222.0.23 () {58 vars in 1232 bytes} [Thu May 19 12:30:40 2022] DELETE /identity/v3/projects/7dd1b178667e4d91ad125c3cbf030de6 => generated 0 bytes in 628 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:30:41.381727 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4081454d-da40-4d2f-90e9-ac281c1a6ae0 admin admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:30:41.385882 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-4081454d-da40-4d2f-90e9-ac281c1a6ae0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:30:41.393797 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1297/2619] 10.222.0.23 () {58 vars in 957 bytes} [Thu May 19 12:30:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 12:30:41.404037 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2a5dc7-cf45-4c07-86a8-643b57afa380 None None] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:30:41.404596 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2a5dc7-cf45-4c07-86a8-643b57afa380 None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:30:41.405099 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2a5dc7-cf45-4c07-86a8-643b57afa380 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:30:41.440302 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.auth.core [None req-8c2a5dc7-cf45-4c07-86a8-643b57afa380 None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:30:41.442595 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-8c2a5dc7-cf45-4c07-86a8-643b57afa380 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:30:41.460244 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1298/2620] 10.222.0.23 () {58 vars in 957 bytes} [Thu May 19 12:30:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 12:30:41.465945 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2a5dc7-cf45-4c07-86a8-643b57afa380 None None] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:30:41.481799 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2a5dc7-cf45-4c07-86a8-643b57afa380 None None] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:30:41.491953 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1939aab4-0226-4034-b058-cdebda5aa158 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:30:41.493334 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1939aab4-0226-4034-b058-cdebda5aa158 admin admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:30:41.493817 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1939aab4-0226-4034-b058-cdebda5aa158 admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:30:41.494288 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1939aab4-0226-4034-b058-cdebda5aa158 admin admin] PATH_INFO: `/v3/domains` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:30:41.496105 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1939aab4-0226-4034-b058-cdebda5aa158 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:30:41.496628 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1939aab4-0226-4034-b058-cdebda5aa158 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:30:41.500701 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1939aab4-0226-4034-b058-cdebda5aa158 admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:30:41.504670 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4081454d-da40-4d2f-90e9-ac281c1a6ae0 admin admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:30:41.508325 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db30cb5-5090-4664-a895-14c46f1d5c37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:30:41.511642 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db30cb5-5090-4664-a895-14c46f1d5c37 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:30:41.511642 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db30cb5-5090-4664-a895-14c46f1d5c37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:30:41.511642 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db30cb5-5090-4664-a895-14c46f1d5c37 admin admin] PATH_INFO: `/v3/users/ce34e262e9c84dfaa58d50b2e643daec` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:30:41.513812 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db30cb5-5090-4664-a895-14c46f1d5c37 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ce34e262e9c84dfaa58d50b2e643daec)` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:30:41.517752 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db30cb5-5090-4664-a895-14c46f1d5c37 admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:30:41.524477 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1299/2621] 10.222.0.23 () {58 vars in 1152 bytes} [Thu May 19 12:30:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 12:30:41.531075 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1939aab4-0226-4034-b058-cdebda5aa158 admin admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:30:41.548821 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1939aab4-0226-4034-b058-cdebda5aa158 admin admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:30:41.556380 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60c2b58-ebd7-4539-9500-db12a1244618 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:30:41.557298 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60c2b58-ebd7-4539-9500-db12a1244618 admin admin] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:30:41.558059 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60c2b58-ebd7-4539-9500-db12a1244618 admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:30:41.558612 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60c2b58-ebd7-4539-9500-db12a1244618 admin admin] PATH_INFO: `/v3/projects` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:30:41.559453 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60c2b58-ebd7-4539-9500-db12a1244618 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:30:41.562779 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60c2b58-ebd7-4539-9500-db12a1244618 admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:30:41.685173 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1300/2622] 10.222.0.23 () {60 vars in 1152 bytes} [Thu May 19 12:30:41 2022] POST /identity/v3/projects => generated 377 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 12:30:41.690482 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60c2b58-ebd7-4539-9500-db12a1244618 admin admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:30:41.786642 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60c2b58-ebd7-4539-9500-db12a1244618 admin admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:30:41.799534 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ae6069-c913-4c11-bbb7-baef12686e8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:30:41.800250 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ae6069-c913-4c11-bbb7-baef12686e8b admin admin] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:30:41.800821 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ae6069-c913-4c11-bbb7-baef12686e8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:30:41.801326 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ae6069-c913-4c11-bbb7-baef12686e8b admin admin] PATH_INFO: `/v3/users` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:30:41.802123 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ae6069-c913-4c11-bbb7-baef12686e8b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:30:41.804299 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ae6069-c913-4c11-bbb7-baef12686e8b admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:30:41.824201 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-2db30cb5-5090-4664-a895-14c46f1d5c37 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ce34e262e9c84dfaa58d50b2e643daec'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:30:41.849104 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-2db30cb5-5090-4664-a895-14c46f1d5c37 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ce34e262e9c84dfaa58d50b2e643daec'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:30:41.850088 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-2db30cb5-5090-4664-a895-14c46f1d5c37 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ce34e262e9c84dfaa58d50b2e643daec'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:30:41.937543 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1301/2623] 10.222.0.23 () {60 vars in 1143 bytes} [Thu May 19 12:30:41 2022] POST /identity/v3/users => generated 289 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 12:30:41.943183 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ae6069-c913-4c11-bbb7-baef12686e8b admin admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:30:41.962246 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ae6069-c913-4c11-bbb7-baef12686e8b admin admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:30:41.969514 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255ab584-b6c2-474a-98d0-cd0505a8bf1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:30:41.973844 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255ab584-b6c2-474a-98d0-cd0505a8bf1c admin admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:30:41.973844 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255ab584-b6c2-474a-98d0-cd0505a8bf1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:30:41.973844 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255ab584-b6c2-474a-98d0-cd0505a8bf1c admin admin] PATH_INFO: `/v3/roles` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:30:41.973844 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255ab584-b6c2-474a-98d0-cd0505a8bf1c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:30:41.973844 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255ab584-b6c2-474a-98d0-cd0505a8bf1c admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:30:41.989494 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1302/2624] 10.222.0.23 () {58 vars in 1121 bytes} [Thu May 19 12:30:41 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 19 12:30:41.996811 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255ab584-b6c2-474a-98d0-cd0505a8bf1c admin admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:30:42.003661 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-2db30cb5-5090-4664-a895-14c46f1d5c37 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ce34e262e9c84dfaa58d50b2e643daec'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:30:42.008610 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:30:42.017264 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255ab584-b6c2-474a-98d0-cd0505a8bf1c admin admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:30:42.022842 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0ba7bc-556b-4c12-8942-b912c924f226 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:30:42.024297 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-2db30cb5-5090-4664-a895-14c46f1d5c37 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ce34e262e9c84dfaa58d50b2e643daec'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:30:42.024499 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0ba7bc-556b-4c12-8942-b912c924f226 admin admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:30:42.024499 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0ba7bc-556b-4c12-8942-b912c924f226 admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:30:42.024499 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0ba7bc-556b-4c12-8942-b912c924f226 admin admin] PATH_INFO: `/v3/roles` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:30:42.026581 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0ba7bc-556b-4c12-8942-b912c924f226 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:30:42.027634 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0ba7bc-556b-4c12-8942-b912c924f226 admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:30:42.036400 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1323/2625] 10.222.0.23 () {58 vars in 1223 bytes} [Thu May 19 12:30:41 2022] DELETE /identity/v3/users/ce34e262e9c84dfaa58d50b2e643daec => generated 0 bytes in 655 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 12:30:42.047271 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1303/2626] 10.222.0.23 () {58 vars in 1121 bytes} [Thu May 19 12:30:41 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 12:30:42.054248 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db30cb5-5090-4664-a895-14c46f1d5c37 admin admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:30:42.057424 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-2db30cb5-5090-4664-a895-14c46f1d5c37 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:30:42.188882 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db30cb5-5090-4664-a895-14c46f1d5c37 admin admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:30:42.194539 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a5e4a4-945f-476d-97ac-580ebdec55e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:30:42.195576 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a5e4a4-945f-476d-97ac-580ebdec55e7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:30:42.195914 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a5e4a4-945f-476d-97ac-580ebdec55e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:30:42.196193 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a5e4a4-945f-476d-97ac-580ebdec55e7 admin admin] PATH_INFO: `/v3/projects/faf9e920a9484c7f99e283ac8579267c/users/ae74c2a7dd774a18816bc1be80de6d92/roles/259eb400cdb4439db72d14456ae9cd08` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:30:42.214339 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a5e4a4-945f-476d-97ac-580ebdec55e7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=faf9e920a9484c7f99e283ac8579267c, user_id=ae74c2a7dd774a18816bc1be80de6d92, role_id=259eb400cdb4439db72d14456ae9cd08)` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:30:42.217241 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a5e4a4-945f-476d-97ac-580ebdec55e7 admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:30:42.301986 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:DELETE=13,keystone:SELECT=85,keystone:INSERT=2 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:30:42.315924 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1324/2627] 10.222.0.23 () {60 vars in 1482 bytes} [Thu May 19 12:30:42 2022] PUT /identity/v3/projects/faf9e920a9484c7f99e283ac8579267c/users/ae74c2a7dd774a18816bc1be80de6d92/roles/259eb400cdb4439db72d14456ae9cd08 => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:30:42.332982 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bf33cc-6d73-4f7e-9fc2-e294baa9f4a1 None None] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:30:42.338408 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bf33cc-6d73-4f7e-9fc2-e294baa9f4a1 None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:30:42.339581 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bf33cc-6d73-4f7e-9fc2-e294baa9f4a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:30:42.414425 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.auth.core [None req-72bf33cc-6d73-4f7e-9fc2-e294baa9f4a1 None None] MFA Rules not processed for user `ae74c2a7dd774a18816bc1be80de6d92`. Rule list: `[]` (Enabled: `True`). {{(pid=80539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:30:42.416066 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-72bf33cc-6d73-4f7e-9fc2-e294baa9f4a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:30:42.547194 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1304/2628] 10.222.0.23 () {58 vars in 957 bytes} [Thu May 19 12:30:42 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:30:42.547865 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a5e4a4-945f-476d-97ac-580ebdec55e7 admin admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:30:42.636244 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a5e4a4-945f-476d-97ac-580ebdec55e7 admin admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:30:42.644467 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2cf3fee-cf9d-4a3f-8d34-136f842f5797 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:30:42.645332 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cf3fee-cf9d-4a3f-8d34-136f842f5797 admin admin] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:30:42.645546 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cf3fee-cf9d-4a3f-8d34-136f842f5797 admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:30:42.645873 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cf3fee-cf9d-4a3f-8d34-136f842f5797 admin admin] PATH_INFO: `/v3/projects` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:30:42.646721 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2cf3fee-cf9d-4a3f-8d34-136f842f5797 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:30:42.648848 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2cf3fee-cf9d-4a3f-8d34-136f842f5797 admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:30:42.657388 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72bf33cc-6d73-4f7e-9fc2-e294baa9f4a1 None None] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:30:42.659248 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-72bf33cc-6d73-4f7e-9fc2-e294baa9f4a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:30:42.687844 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72bf33cc-6d73-4f7e-9fc2-e294baa9f4a1 None None] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:30:42.694530 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07dbfbd-23b1-4e67-b5ff-39e78fd334df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:30:42.696138 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07dbfbd-23b1-4e67-b5ff-39e78fd334df admin admin] REQUEST_METHOD: `DELETE` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:30:42.697090 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07dbfbd-23b1-4e67-b5ff-39e78fd334df admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:30:42.697622 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07dbfbd-23b1-4e67-b5ff-39e78fd334df admin admin] PATH_INFO: `/v3/projects/0c5478cf18984859ad9fd5f501afe0da` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:30:42.700883 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07dbfbd-23b1-4e67-b5ff-39e78fd334df admin admin] RBAC: Authorizing `identity:delete_project(project_id=0c5478cf18984859ad9fd5f501afe0da)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:30:42.702949 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07dbfbd-23b1-4e67-b5ff-39e78fd334df admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:30:42.750587 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1325/2629] 10.222.0.23 () {60 vars in 1152 bytes} [Thu May 19 12:30:42 2022] POST /identity/v3/projects => generated 375 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 12:30:42.754719 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2cf3fee-cf9d-4a3f-8d34-136f842f5797 admin admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:30:42.850341 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2cf3fee-cf9d-4a3f-8d34-136f842f5797 admin admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:30:42.856594 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f44a1a-1fa4-40b5-8a8d-9cae949fb33b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:30:42.857013 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f44a1a-1fa4-40b5-8a8d-9cae949fb33b admin admin] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:30:42.857013 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f44a1a-1fa4-40b5-8a8d-9cae949fb33b admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:30:42.857271 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f44a1a-1fa4-40b5-8a8d-9cae949fb33b admin admin] PATH_INFO: `/v3/users` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:30:42.858017 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f44a1a-1fa4-40b5-8a8d-9cae949fb33b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:30:42.860375 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f44a1a-1fa4-40b5-8a8d-9cae949fb33b admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:30:42.889934 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-b07dbfbd-23b1-4e67-b5ff-39e78fd334df admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0c5478cf18984859ad9fd5f501afe0da', 'request_id': 'req-b07dbfbd-23b1-4e67-b5ff-39e78fd334df'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:30:42.903744 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-b07dbfbd-23b1-4e67-b5ff-39e78fd334df admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0c5478cf18984859ad9fd5f501afe0da', 'request_id': 'req-b07dbfbd-23b1-4e67-b5ff-39e78fd334df'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:30:43.114741 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-b07dbfbd-23b1-4e67-b5ff-39e78fd334df admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0c5478cf18984859ad9fd5f501afe0da', 'request_id': 'req-b07dbfbd-23b1-4e67-b5ff-39e78fd334df'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:30:43.126537 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1326/2630] 10.222.0.23 () {60 vars in 1143 bytes} [Thu May 19 12:30:42 2022] POST /identity/v3/users => generated 288 bytes in 372 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 12:30:43.133239 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f44a1a-1fa4-40b5-8a8d-9cae949fb33b admin admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:30:43.143943 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-b07dbfbd-23b1-4e67-b5ff-39e78fd334df admin admin] The token cache is being invalidate because project 0c5478cf18984859ad9fd5f501afe0da was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80539) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 12:30:43.146704 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-b07dbfbd-23b1-4e67-b5ff-39e78fd334df admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:30:43.148737 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-b07dbfbd-23b1-4e67-b5ff-39e78fd334df admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:30:43.158224 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1305/2631] 10.222.0.23 () {58 vars in 1232 bytes} [Thu May 19 12:30:42 2022] DELETE /identity/v3/projects/0c5478cf18984859ad9fd5f501afe0da => generated 0 bytes in 500 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:30:43.169865 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f44a1a-1fa4-40b5-8a8d-9cae949fb33b admin admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:30:43.172480 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9f3694-4477-48ae-bff9-3efb4f64cfff None None] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:30:43.173166 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9f3694-4477-48ae-bff9-3efb4f64cfff None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:30:43.173769 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9f3694-4477-48ae-bff9-3efb4f64cfff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:30:43.207143 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.auth.core [None req-4a9f3694-4477-48ae-bff9-3efb4f64cfff None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:30:43.209313 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-4a9f3694-4477-48ae-bff9-3efb4f64cfff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:30:43.302002 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602abeaa-8fee-4bfd-823f-9f0b322f4168 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:30:43.303591 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602abeaa-8fee-4bfd-823f-9f0b322f4168 admin admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:30:43.304159 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602abeaa-8fee-4bfd-823f-9f0b322f4168 admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:30:43.304637 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602abeaa-8fee-4bfd-823f-9f0b322f4168 admin admin] PATH_INFO: `/v3/roles` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:30:43.306187 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602abeaa-8fee-4bfd-823f-9f0b322f4168 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:30:43.310150 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602abeaa-8fee-4bfd-823f-9f0b322f4168 admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:30:43.326827 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:30:43.343151 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1327/2632] 10.222.0.23 () {58 vars in 1121 bytes} [Thu May 19 12:30:43 2022] GET /identity/v3/roles => generated 1420 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 12:30:43.352339 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602abeaa-8fee-4bfd-823f-9f0b322f4168 admin admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:30:43.358162 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-602abeaa-8fee-4bfd-823f-9f0b322f4168 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:30:43.368324 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1306/2633] 10.222.0.23 () {58 vars in 957 bytes} [Thu May 19 12:30:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:30:43.386069 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602abeaa-8fee-4bfd-823f-9f0b322f4168 admin admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:30:43.388693 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b97afe-2cc2-49c3-b65b-73b9cab4690a None None] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:30:43.389098 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b97afe-2cc2-49c3-b65b-73b9cab4690a None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:30:43.389481 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b97afe-2cc2-49c3-b65b-73b9cab4690a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:30:43.408053 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea51851f-2b1f-4eed-8376-397c15f40e91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:30:43.408928 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea51851f-2b1f-4eed-8376-397c15f40e91 admin admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:30:43.415006 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea51851f-2b1f-4eed-8376-397c15f40e91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:30:43.415006 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea51851f-2b1f-4eed-8376-397c15f40e91 admin admin] PATH_INFO: `/v3/roles` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:30:43.415006 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea51851f-2b1f-4eed-8376-397c15f40e91 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:30:43.415006 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea51851f-2b1f-4eed-8376-397c15f40e91 admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:30:43.419145 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.auth.core [None req-77b97afe-2cc2-49c3-b65b-73b9cab4690a None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:30:43.420648 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-77b97afe-2cc2-49c3-b65b-73b9cab4690a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:30:43.435446 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1328/2634] 10.222.0.23 () {58 vars in 1121 bytes} [Thu May 19 12:30:43 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 12:30:43.435765 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1307/2635] 10.222.0.23 () {58 vars in 957 bytes} [Thu May 19 12:30:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 12:30:43.436909 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea51851f-2b1f-4eed-8376-397c15f40e91 admin admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:30:43.438143 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b97afe-2cc2-49c3-b65b-73b9cab4690a None None] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:30:43.450211 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:30:43.450863 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b97afe-2cc2-49c3-b65b-73b9cab4690a None None] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:30:43.457964 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f507856f-7b4f-4264-9275-1eaeac625cfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:30:43.458987 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f507856f-7b4f-4264-9275-1eaeac625cfb admin admin] REQUEST_METHOD: `PUT` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:30:43.459280 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f507856f-7b4f-4264-9275-1eaeac625cfb admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:30:43.459741 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f507856f-7b4f-4264-9275-1eaeac625cfb admin admin] PATH_INFO: `/v3/projects/fbc3a9ef12194870b924c6590b91fc1b/users/e48965a7840448b6b6c8524137f6edf0/roles/259eb400cdb4439db72d14456ae9cd08` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:30:43.469615 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea51851f-2b1f-4eed-8376-397c15f40e91 admin admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:30:43.479162 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f507856f-7b4f-4264-9275-1eaeac625cfb admin admin] RBAC: Authorizing `identity:create_grant(project_id=fbc3a9ef12194870b924c6590b91fc1b, user_id=e48965a7840448b6b6c8524137f6edf0, role_id=259eb400cdb4439db72d14456ae9cd08)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:30:43.482874 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f507856f-7b4f-4264-9275-1eaeac625cfb admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:30:43.484841 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6cfad1e-2039-4941-92e0-05659b1cb62c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:30:43.487433 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cfad1e-2039-4941-92e0-05659b1cb62c admin admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:30:43.487728 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cfad1e-2039-4941-92e0-05659b1cb62c admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:30:43.488104 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cfad1e-2039-4941-92e0-05659b1cb62c admin admin] PATH_INFO: `/v3/domains` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:30:43.489255 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6cfad1e-2039-4941-92e0-05659b1cb62c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:30:43.489761 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6cfad1e-2039-4941-92e0-05659b1cb62c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:30:43.491665 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6cfad1e-2039-4941-92e0-05659b1cb62c admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:30:43.513656 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1329/2636] 10.222.0.23 () {58 vars in 1152 bytes} [Thu May 19 12:30:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:30:43.517824 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6cfad1e-2039-4941-92e0-05659b1cb62c admin admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:30:43.530025 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6cfad1e-2039-4941-92e0-05659b1cb62c admin admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:30:43.538824 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed46862f-7879-484d-aaa4-128624a7a845 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:30:43.539566 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed46862f-7879-484d-aaa4-128624a7a845 admin admin] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:30:43.539897 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed46862f-7879-484d-aaa4-128624a7a845 admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:30:43.540188 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed46862f-7879-484d-aaa4-128624a7a845 admin admin] PATH_INFO: `/v3/projects` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:30:43.540908 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed46862f-7879-484d-aaa4-128624a7a845 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:30:43.544716 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed46862f-7879-484d-aaa4-128624a7a845 admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:30:43.654481 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1308/2637] 10.222.0.23 () {60 vars in 1482 bytes} [Thu May 19 12:30:43 2022] PUT /identity/v3/projects/fbc3a9ef12194870b924c6590b91fc1b/users/e48965a7840448b6b6c8524137f6edf0/roles/259eb400cdb4439db72d14456ae9cd08 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 12:30:43.668811 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfbbec0-82a1-44d5-a07a-10d7ae6b44d1 None None] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:30:43.669379 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfbbec0-82a1-44d5-a07a-10d7ae6b44d1 None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:30:43.669850 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfbbec0-82a1-44d5-a07a-10d7ae6b44d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:30:43.701616 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1330/2638] 10.222.0.23 () {60 vars in 1152 bytes} [Thu May 19 12:30:43 2022] POST /identity/v3/projects => generated 381 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 12:30:43.703278 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed46862f-7879-484d-aaa4-128624a7a845 admin admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:30:43.761993 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.auth.core [None req-8dfbbec0-82a1-44d5-a07a-10d7ae6b44d1 None None] MFA Rules not processed for user `e48965a7840448b6b6c8524137f6edf0`. Rule list: `[]` (Enabled: `True`). {{(pid=80539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:30:43.764150 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-8dfbbec0-82a1-44d5-a07a-10d7ae6b44d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:30:43.941925 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed46862f-7879-484d-aaa4-128624a7a845 admin admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:30:43.980626 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0156dc3f-a272-487b-aac0-bcdb892c8c05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:30:43.981839 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0156dc3f-a272-487b-aac0-bcdb892c8c05 admin admin] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:30:43.982306 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0156dc3f-a272-487b-aac0-bcdb892c8c05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:30:43.982823 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0156dc3f-a272-487b-aac0-bcdb892c8c05 admin admin] PATH_INFO: `/v3/users` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:30:43.986462 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0156dc3f-a272-487b-aac0-bcdb892c8c05 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:30:43.997025 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0156dc3f-a272-487b-aac0-bcdb892c8c05 admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:30:43.997183 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1309/2639] 10.222.0.23 () {58 vars in 957 bytes} [Thu May 19 12:30:43 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 328 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:30:43.999260 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfbbec0-82a1-44d5-a07a-10d7ae6b44d1 None None] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:30:44.017386 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfbbec0-82a1-44d5-a07a-10d7ae6b44d1 None None] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:30:44.027492 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775bbb05-62ac-4bfd-a00e-45b5d840bf0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:30:44.028850 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775bbb05-62ac-4bfd-a00e-45b5d840bf0c admin admin] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:30:44.029325 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775bbb05-62ac-4bfd-a00e-45b5d840bf0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:30:44.029959 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775bbb05-62ac-4bfd-a00e-45b5d840bf0c admin admin] PATH_INFO: `/v3/projects` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:30:44.031395 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775bbb05-62ac-4bfd-a00e-45b5d840bf0c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:30:44.035229 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775bbb05-62ac-4bfd-a00e-45b5d840bf0c admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:30:44.123195 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1331/2640] 10.222.0.23 () {60 vars in 1143 bytes} [Thu May 19 12:30:43 2022] POST /identity/v3/users => generated 291 bytes in 420 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 12:30:44.125130 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0156dc3f-a272-487b-aac0-bcdb892c8c05 admin admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:30:44.159541 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0156dc3f-a272-487b-aac0-bcdb892c8c05 admin admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:30:44.176370 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a456e3-a8b8-4dd7-a5f2-2796efe02887 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:30:44.180358 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a456e3-a8b8-4dd7-a5f2-2796efe02887 admin admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:30:44.180358 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a456e3-a8b8-4dd7-a5f2-2796efe02887 admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:30:44.180358 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a456e3-a8b8-4dd7-a5f2-2796efe02887 admin admin] PATH_INFO: `/v3/roles` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:30:44.180975 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a456e3-a8b8-4dd7-a5f2-2796efe02887 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:30:44.187001 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a456e3-a8b8-4dd7-a5f2-2796efe02887 admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:30:44.215133 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1332/2641] 10.222.0.23 () {58 vars in 1121 bytes} [Thu May 19 12:30:44 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 12:30:44.221158 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a456e3-a8b8-4dd7-a5f2-2796efe02887 admin admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:30:44.236345 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a456e3-a8b8-4dd7-a5f2-2796efe02887 admin admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:30:44.254394 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1310/2642] 10.222.0.23 () {60 vars in 1152 bytes} [Thu May 19 12:30:43 2022] POST /identity/v3/projects => generated 377 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 12:30:44.267176 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775bbb05-62ac-4bfd-a00e-45b5d840bf0c admin admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:30:44.446276 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345b796f-5280-488f-b698-ea5d00a86b88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:30:44.447964 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345b796f-5280-488f-b698-ea5d00a86b88 admin admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:30:44.448534 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345b796f-5280-488f-b698-ea5d00a86b88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:30:44.448985 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345b796f-5280-488f-b698-ea5d00a86b88 admin admin] PATH_INFO: `/v3/roles` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:30:44.450049 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345b796f-5280-488f-b698-ea5d00a86b88 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:30:44.452368 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345b796f-5280-488f-b698-ea5d00a86b88 admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:30:44.474421 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1333/2643] 10.222.0.23 () {58 vars in 1121 bytes} [Thu May 19 12:30:44 2022] GET /identity/v3/roles => generated 1420 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 19 12:30:44.481678 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345b796f-5280-488f-b698-ea5d00a86b88 admin admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:30:44.524162 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345b796f-5280-488f-b698-ea5d00a86b88 admin admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:30:44.537612 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a7ad5c-a4d1-42ed-8c0a-3c1da631c06d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:30:44.539218 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a7ad5c-a4d1-42ed-8c0a-3c1da631c06d admin admin] REQUEST_METHOD: `PUT` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:30:44.540031 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:30:44.540248 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a7ad5c-a4d1-42ed-8c0a-3c1da631c06d admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:30:44.540549 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a7ad5c-a4d1-42ed-8c0a-3c1da631c06d admin admin] PATH_INFO: `/v3/projects/b4fd0bea1cd04935862d0f74cf5be84c/users/7c222658b1074dc4b6db18f9760a2099/roles/259eb400cdb4439db72d14456ae9cd08` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:30:44.552503 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775bbb05-62ac-4bfd-a00e-45b5d840bf0c admin admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:30:44.579092 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a7ad5c-a4d1-42ed-8c0a-3c1da631c06d admin admin] RBAC: Authorizing `identity:create_grant(project_id=b4fd0bea1cd04935862d0f74cf5be84c, user_id=7c222658b1074dc4b6db18f9760a2099, role_id=259eb400cdb4439db72d14456ae9cd08)` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:30:44.579842 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91762b0-2b36-4b05-a8eb-07915dcbdf71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:30:44.581337 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91762b0-2b36-4b05-a8eb-07915dcbdf71 admin admin] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:30:44.584841 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a7ad5c-a4d1-42ed-8c0a-3c1da631c06d admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:30:44.586938 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91762b0-2b36-4b05-a8eb-07915dcbdf71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:30:44.590007 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91762b0-2b36-4b05-a8eb-07915dcbdf71 admin admin] PATH_INFO: `/v3/users` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:30:44.591876 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91762b0-2b36-4b05-a8eb-07915dcbdf71 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:30:44.602644 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91762b0-2b36-4b05-a8eb-07915dcbdf71 admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:30:44.686280 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:30:44.700111 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1334/2644] 10.222.0.23 () {60 vars in 1482 bytes} [Thu May 19 12:30:44 2022] PUT /identity/v3/projects/b4fd0bea1cd04935862d0f74cf5be84c/users/7c222658b1074dc4b6db18f9760a2099/roles/259eb400cdb4439db72d14456ae9cd08 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:30:44.721371 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97d3478-5af0-455e-9a77-a35a7661f35a None None] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:30:44.722266 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97d3478-5af0-455e-9a77-a35a7661f35a None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:30:44.722438 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97d3478-5af0-455e-9a77-a35a7661f35a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:30:44.749716 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1311/2645] 10.222.0.23 () {60 vars in 1143 bytes} [Thu May 19 12:30:44 2022] POST /identity/v3/users => generated 295 bytes in 495 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 12:30:44.754838 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91762b0-2b36-4b05-a8eb-07915dcbdf71 admin admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:30:44.778312 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91762b0-2b36-4b05-a8eb-07915dcbdf71 admin admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:30:44.784934 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.auth.core [None req-b97d3478-5af0-455e-9a77-a35a7661f35a None None] MFA Rules not processed for user `7c222658b1074dc4b6db18f9760a2099`. Rule list: `[]` (Enabled: `True`). {{(pid=80540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:30:44.784934 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-b97d3478-5af0-455e-9a77-a35a7661f35a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:30:44.913897 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1335/2646] 10.222.0.23 () {58 vars in 957 bytes} [Thu May 19 12:30:44 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 12:30:44.919809 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97d3478-5af0-455e-9a77-a35a7661f35a None None] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:30:44.929937 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac1d3fb-9b2b-4b06-a78a-70ccfaef91fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:30:44.930934 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac1d3fb-9b2b-4b06-a78a-70ccfaef91fd admin admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:30:44.931491 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac1d3fb-9b2b-4b06-a78a-70ccfaef91fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:30:44.932300 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac1d3fb-9b2b-4b06-a78a-70ccfaef91fd admin admin] PATH_INFO: `/v3/roles` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:30:44.933439 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac1d3fb-9b2b-4b06-a78a-70ccfaef91fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:30:44.935130 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac1d3fb-9b2b-4b06-a78a-70ccfaef91fd admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:30:44.962843 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1312/2647] 10.222.0.23 () {58 vars in 1121 bytes} [Thu May 19 12:30:44 2022] GET /identity/v3/roles => generated 1420 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 12:30:44.966932 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac1d3fb-9b2b-4b06-a78a-70ccfaef91fd admin admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:30:44.999687 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac1d3fb-9b2b-4b06-a78a-70ccfaef91fd admin admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:30:45.016101 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16dc631b-7ee7-4f64-88d5-09223b83009a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:30:45.018849 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16dc631b-7ee7-4f64-88d5-09223b83009a admin admin] REQUEST_METHOD: `PUT` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:30:45.019635 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16dc631b-7ee7-4f64-88d5-09223b83009a admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:30:45.022731 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16dc631b-7ee7-4f64-88d5-09223b83009a admin admin] PATH_INFO: `/v3/projects/9575ff3f86604308bc12b4b2cc6ff27a/users/8bc14ec9db0042e8a51a27d401bd7a76/roles/9db5671c65d3414ea2682afa2ccf1403` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:30:45.057669 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16dc631b-7ee7-4f64-88d5-09223b83009a admin admin] RBAC: Authorizing `identity:create_grant(project_id=9575ff3f86604308bc12b4b2cc6ff27a, user_id=8bc14ec9db0042e8a51a27d401bd7a76, role_id=9db5671c65d3414ea2682afa2ccf1403)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:30:45.064354 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16dc631b-7ee7-4f64-88d5-09223b83009a admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:30:45.093332 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97d3478-5af0-455e-9a77-a35a7661f35a None None] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:30:45.109623 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7c22d5-e6f5-4ba6-85d6-c05fb1b4b92b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:30:45.113362 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7c22d5-e6f5-4ba6-85d6-c05fb1b4b92b admin admin] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:30:45.113362 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7c22d5-e6f5-4ba6-85d6-c05fb1b4b92b admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:30:45.113362 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7c22d5-e6f5-4ba6-85d6-c05fb1b4b92b admin admin] PATH_INFO: `/v3/projects` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:30:45.115855 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7c22d5-e6f5-4ba6-85d6-c05fb1b4b92b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:30:45.133932 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7c22d5-e6f5-4ba6-85d6-c05fb1b4b92b admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:30:45.193207 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1313/2648] 10.222.0.23 () {60 vars in 1482 bytes} [Thu May 19 12:30:44 2022] PUT /identity/v3/projects/9575ff3f86604308bc12b4b2cc6ff27a/users/8bc14ec9db0042e8a51a27d401bd7a76/roles/9db5671c65d3414ea2682afa2ccf1403 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 12:30:45.198751 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16dc631b-7ee7-4f64-88d5-09223b83009a admin admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:30:45.261240 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1336/2649] 10.222.0.23 () {60 vars in 1152 bytes} [Thu May 19 12:30:44 2022] POST /identity/v3/projects => generated 379 bytes in 342 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 12:30:45.267680 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7c22d5-e6f5-4ba6-85d6-c05fb1b4b92b admin admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:30:45.401584 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16dc631b-7ee7-4f64-88d5-09223b83009a admin admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:30:45.435836 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7c22d5-e6f5-4ba6-85d6-c05fb1b4b92b admin admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:30:45.444538 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698a7720-010e-4bfd-9972-311b903e3a9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:30:45.446044 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698a7720-010e-4bfd-9972-311b903e3a9f admin admin] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:30:45.446373 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698a7720-010e-4bfd-9972-311b903e3a9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:30:45.446971 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698a7720-010e-4bfd-9972-311b903e3a9f admin admin] PATH_INFO: `/v3/users` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:30:45.449358 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698a7720-010e-4bfd-9972-311b903e3a9f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:30:45.454385 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698a7720-010e-4bfd-9972-311b903e3a9f admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:30:45.575862 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1337/2650] 10.222.0.23 () {60 vars in 1143 bytes} [Thu May 19 12:30:45 2022] POST /identity/v3/users => generated 296 bytes in 307 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 12:30:45.578739 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698a7720-010e-4bfd-9972-311b903e3a9f admin admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:30:45.581059 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691bf8d2-d356-4682-96f4-f04350d82ae0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:30:45.582219 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691bf8d2-d356-4682-96f4-f04350d82ae0 admin admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:30:45.586055 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691bf8d2-d356-4682-96f4-f04350d82ae0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:30:45.587380 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:30:45.588732 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691bf8d2-d356-4682-96f4-f04350d82ae0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:30:45.593015 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691bf8d2-d356-4682-96f4-f04350d82ae0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:30:45.599579 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691bf8d2-d356-4682-96f4-f04350d82ae0 admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:30:45.610478 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698a7720-010e-4bfd-9972-311b903e3a9f admin admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:30:45.618360 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:30:45.629906 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ebe009-7dcc-49af-8840-12b638356bce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:30:45.630972 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ebe009-7dcc-49af-8840-12b638356bce admin admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:30:45.633480 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ebe009-7dcc-49af-8840-12b638356bce admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:30:45.633480 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ebe009-7dcc-49af-8840-12b638356bce admin admin] PATH_INFO: `/v3/roles` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:30:45.633480 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ebe009-7dcc-49af-8840-12b638356bce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:30:45.637144 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ebe009-7dcc-49af-8840-12b638356bce admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:30:45.640603 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1314/2651] 10.222.0.23 () {58 vars in 1121 bytes} [Thu May 19 12:30:45 2022] GET /identity/v3/roles => generated 1420 bytes in 443 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 12:30:45.642404 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691bf8d2-d356-4682-96f4-f04350d82ae0 admin admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:30:45.664017 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1338/2652] 10.222.0.23 () {58 vars in 1121 bytes} [Thu May 19 12:30:45 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 19 12:30:45.670223 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691bf8d2-d356-4682-96f4-f04350d82ae0 admin admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:30:45.673705 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ebe009-7dcc-49af-8840-12b638356bce admin admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:30:45.680771 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42f1fe2-0aa8-49b8-883e-afb57d06f1e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:30:45.683097 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42f1fe2-0aa8-49b8-883e-afb57d06f1e9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:30:45.683664 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42f1fe2-0aa8-49b8-883e-afb57d06f1e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:30:45.684147 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42f1fe2-0aa8-49b8-883e-afb57d06f1e9 admin admin] PATH_INFO: `/v3/projects/9575ff3f86604308bc12b4b2cc6ff27a/users/8bc14ec9db0042e8a51a27d401bd7a76/roles/9db5671c65d3414ea2682afa2ccf1403` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:30:45.689528 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42f1fe2-0aa8-49b8-883e-afb57d06f1e9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9575ff3f86604308bc12b4b2cc6ff27a, user_id=8bc14ec9db0042e8a51a27d401bd7a76, role_id=9db5671c65d3414ea2682afa2ccf1403)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:30:45.697986 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42f1fe2-0aa8-49b8-883e-afb57d06f1e9 admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:30:45.709986 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ebe009-7dcc-49af-8840-12b638356bce admin admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:30:45.722352 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3507c53f-101c-4b9c-ac4c-4b0d2f3d868e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:30:45.725956 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3507c53f-101c-4b9c-ac4c-4b0d2f3d868e admin admin] REQUEST_METHOD: `PUT` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:30:45.725956 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3507c53f-101c-4b9c-ac4c-4b0d2f3d868e admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:30:45.725956 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3507c53f-101c-4b9c-ac4c-4b0d2f3d868e admin admin] PATH_INFO: `/v3/projects/b1c907159a9646cfab27300decf709d1/users/894a787687384f14be2c5cf412715100/roles/9db5671c65d3414ea2682afa2ccf1403` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:30:45.728688 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1315/2653] 10.222.0.23 () {60 vars in 1482 bytes} [Thu May 19 12:30:45 2022] PUT /identity/v3/projects/9575ff3f86604308bc12b4b2cc6ff27a/users/8bc14ec9db0042e8a51a27d401bd7a76/roles/9db5671c65d3414ea2682afa2ccf1403 => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:30:45.749905 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75773610-2186-4b13-9549-fbbd94e213ec None None] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:30:45.750751 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75773610-2186-4b13-9549-fbbd94e213ec None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:30:45.752025 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75773610-2186-4b13-9549-fbbd94e213ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:30:45.790759 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3507c53f-101c-4b9c-ac4c-4b0d2f3d868e admin admin] RBAC: Authorizing `identity:create_grant(project_id=b1c907159a9646cfab27300decf709d1, user_id=894a787687384f14be2c5cf412715100, role_id=9db5671c65d3414ea2682afa2ccf1403)` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:30:45.800173 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3507c53f-101c-4b9c-ac4c-4b0d2f3d868e admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:30:45.801138 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.auth.core [None req-75773610-2186-4b13-9549-fbbd94e213ec None None] MFA Rules not processed for user `8bc14ec9db0042e8a51a27d401bd7a76`. Rule list: `[]` (Enabled: `True`). {{(pid=80539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:30:45.802634 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-75773610-2186-4b13-9549-fbbd94e213ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:30:45.936249 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1339/2654] 10.222.0.23 () {60 vars in 1482 bytes} [Thu May 19 12:30:45 2022] PUT /identity/v3/projects/b1c907159a9646cfab27300decf709d1/users/894a787687384f14be2c5cf412715100/roles/9db5671c65d3414ea2682afa2ccf1403 => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:30:45.942543 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3507c53f-101c-4b9c-ac4c-4b0d2f3d868e admin admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:30:45.970063 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1316/2655] 10.222.0.23 () {58 vars in 957 bytes} [Thu May 19 12:30:45 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:30:45.995579 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75773610-2186-4b13-9549-fbbd94e213ec None None] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:30:46.001452 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-75773610-2186-4b13-9549-fbbd94e213ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:30:46.104183 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3507c53f-101c-4b9c-ac4c-4b0d2f3d868e admin admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:30:46.110583 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6e9afd-a78b-4be7-ab95-b4da1fd29451 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:30:46.111287 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6e9afd-a78b-4be7-ab95-b4da1fd29451 admin admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:30:46.111570 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6e9afd-a78b-4be7-ab95-b4da1fd29451 admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:30:46.111888 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6e9afd-a78b-4be7-ab95-b4da1fd29451 admin admin] PATH_INFO: `/v3/roles` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:30:46.112627 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75773610-2186-4b13-9549-fbbd94e213ec None None] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:30:46.112893 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6e9afd-a78b-4be7-ab95-b4da1fd29451 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:30:46.114672 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6e9afd-a78b-4be7-ab95-b4da1fd29451 admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:30:46.133213 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1340/2656] 10.222.0.23 () {58 vars in 1121 bytes} [Thu May 19 12:30:45 2022] GET /identity/v3/roles => generated 1420 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 19 12:30:46.138455 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6e9afd-a78b-4be7-ab95-b4da1fd29451 admin admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:30:46.149608 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6e9afd-a78b-4be7-ab95-b4da1fd29451 admin admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:30:46.155679 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9160d93c-0e9e-4b30-b507-86f750fe8c87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:30:46.156627 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9160d93c-0e9e-4b30-b507-86f750fe8c87 admin admin] REQUEST_METHOD: `PUT` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:30:46.156905 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9160d93c-0e9e-4b30-b507-86f750fe8c87 admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:30:46.157197 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9160d93c-0e9e-4b30-b507-86f750fe8c87 admin admin] PATH_INFO: `/v3/projects/b1c907159a9646cfab27300decf709d1/users/894a787687384f14be2c5cf412715100/roles/9db5671c65d3414ea2682afa2ccf1403` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:30:46.159430 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9160d93c-0e9e-4b30-b507-86f750fe8c87 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b1c907159a9646cfab27300decf709d1, user_id=894a787687384f14be2c5cf412715100, role_id=9db5671c65d3414ea2682afa2ccf1403)` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:30:46.162199 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9160d93c-0e9e-4b30-b507-86f750fe8c87 admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:30:46.177045 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1341/2657] 10.222.0.23 () {60 vars in 1482 bytes} [Thu May 19 12:30:46 2022] PUT /identity/v3/projects/b1c907159a9646cfab27300decf709d1/users/894a787687384f14be2c5cf412715100/roles/9db5671c65d3414ea2682afa2ccf1403 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 12:30:46.190675 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbad65f-d090-4a5e-867c-375ff11a3239 None None] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:30:46.191168 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbad65f-d090-4a5e-867c-375ff11a3239 None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:30:46.191658 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbad65f-d090-4a5e-867c-375ff11a3239 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:30:46.216991 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c3f82a-42ab-4cc8-be6a-187cee3d8f6b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8800254c0e784e42a791fb9abc274d79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd70d25bf69314820b8aaecb65576978a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:30:46.218058 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c3f82a-42ab-4cc8-be6a-187cee3d8f6b service cinder] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:30:46.218554 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c3f82a-42ab-4cc8-be6a-187cee3d8f6b service cinder] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:30:46.219189 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c3f82a-42ab-4cc8-be6a-187cee3d8f6b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:30:46.222465 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-09c3f82a-42ab-4cc8-be6a-187cee3d8f6b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:30:46.244297 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.auth.core [None req-fdbad65f-d090-4a5e-867c-375ff11a3239 None None] MFA Rules not processed for user `894a787687384f14be2c5cf412715100`. Rule list: `[]` (Enabled: `True`). {{(pid=80540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:30:46.249927 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-fdbad65f-d090-4a5e-867c-375ff11a3239 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:30:46.305068 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:30:46.331382 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c3f82a-42ab-4cc8-be6a-187cee3d8f6b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:30:46.333562 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c3f82a-42ab-4cc8-be6a-187cee3d8f6b service cinder] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:30:46.346538 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1317/2658] 10.222.0.23 () {60 vars in 1365 bytes} [Thu May 19 12:30:45 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:30:46.388169 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1342/2659] 10.222.0.23 () {58 vars in 957 bytes} [Thu May 19 12:30:46 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 12:30:46.405223 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c3f82a-42ab-4cc8-be6a-187cee3d8f6b service cinder] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:30:46.407414 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-09c3f82a-42ab-4cc8-be6a-187cee3d8f6b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:30:46.423899 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c3f82a-42ab-4cc8-be6a-187cee3d8f6b service cinder] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:30:46.492070 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca8b48f-bac4-4464-aee2-2a54fddd8d14 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8800254c0e784e42a791fb9abc274d79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd70d25bf69314820b8aaecb65576978a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:30:46.493264 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca8b48f-bac4-4464-aee2-2a54fddd8d14 service cinder] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:30:46.493264 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca8b48f-bac4-4464-aee2-2a54fddd8d14 service cinder] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:30:46.493264 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca8b48f-bac4-4464-aee2-2a54fddd8d14 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:30:46.506993 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca8b48f-bac4-4464-aee2-2a54fddd8d14 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:30:46.509872 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca8b48f-bac4-4464-aee2-2a54fddd8d14 service cinder] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:30:46.534318 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1318/2660] 10.222.0.23 () {60 vars in 1365 bytes} [Thu May 19 12:30:46 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 12:30:48.014379 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbad65f-d090-4a5e-867c-375ff11a3239 None None] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:30:48.031596 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbad65f-d090-4a5e-867c-375ff11a3239 None None] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:30:48.041653 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb51a295-f6fa-4424-9e76-78dfa6978252 tempest-VolumeQuotaClassesTest-305704263 tempest-VolumeQuotaClassesTest-305704263-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '894a787687384f14be2c5cf412715100', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1c907159a9646cfab27300decf709d1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:30:48.041823 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb51a295-f6fa-4424-9e76-78dfa6978252 tempest-VolumeQuotaClassesTest-305704263 tempest-VolumeQuotaClassesTest-305704263-project-admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:30:48.043595 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb51a295-f6fa-4424-9e76-78dfa6978252 tempest-VolumeQuotaClassesTest-305704263 tempest-VolumeQuotaClassesTest-305704263-project-admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:30:48.043595 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb51a295-f6fa-4424-9e76-78dfa6978252 tempest-VolumeQuotaClassesTest-305704263 tempest-VolumeQuotaClassesTest-305704263-project-admin] PATH_INFO: `/v3/domains` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:30:48.045387 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb51a295-f6fa-4424-9e76-78dfa6978252 tempest-VolumeQuotaClassesTest-305704263 tempest-VolumeQuotaClassesTest-305704263-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:30:48.045903 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb51a295-f6fa-4424-9e76-78dfa6978252 tempest-VolumeQuotaClassesTest-305704263 tempest-VolumeQuotaClassesTest-305704263-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:30:48.048681 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb51a295-f6fa-4424-9e76-78dfa6978252 tempest-VolumeQuotaClassesTest-305704263 tempest-VolumeQuotaClassesTest-305704263-project-admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:30:48.080502 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1343/2661] 10.222.0.23 () {58 vars in 1152 bytes} [Thu May 19 12:30:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:30:48.088184 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca8b48f-bac4-4464-aee2-2a54fddd8d14 service cinder] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:30:48.107086 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca8b48f-bac4-4464-aee2-2a54fddd8d14 service cinder] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:30:48.115714 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff2d6c7-2288-4d88-a077-8a300d167da6 tempest-VolumeQuotaClassesTest-305704263 tempest-VolumeQuotaClassesTest-305704263-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '894a787687384f14be2c5cf412715100', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1c907159a9646cfab27300decf709d1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:30:48.116565 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff2d6c7-2288-4d88-a077-8a300d167da6 tempest-VolumeQuotaClassesTest-305704263 tempest-VolumeQuotaClassesTest-305704263-project-admin] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:30:48.116941 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff2d6c7-2288-4d88-a077-8a300d167da6 tempest-VolumeQuotaClassesTest-305704263 tempest-VolumeQuotaClassesTest-305704263-project-admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:30:48.117729 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff2d6c7-2288-4d88-a077-8a300d167da6 tempest-VolumeQuotaClassesTest-305704263 tempest-VolumeQuotaClassesTest-305704263-project-admin] PATH_INFO: `/v3/projects` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:30:48.118142 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff2d6c7-2288-4d88-a077-8a300d167da6 tempest-VolumeQuotaClassesTest-305704263 tempest-VolumeQuotaClassesTest-305704263-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:30:48.122394 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff2d6c7-2288-4d88-a077-8a300d167da6 tempest-VolumeQuotaClassesTest-305704263 tempest-VolumeQuotaClassesTest-305704263-project-admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:30:48.285823 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1319/2662] 10.222.0.23 () {60 vars in 1152 bytes} [Thu May 19 12:30:48 2022] POST /identity/v3/projects => generated 355 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 12:30:48.290411 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb51a295-f6fa-4424-9e76-78dfa6978252 tempest-VolumeQuotaClassesTest-305704263 tempest-VolumeQuotaClassesTest-305704263-project-admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:30:48.301332 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb51a295-f6fa-4424-9e76-78dfa6978252 tempest-VolumeQuotaClassesTest-305704263 tempest-VolumeQuotaClassesTest-305704263-project-admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:30:48.326256 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:30:48.374610 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef41e174-bd7e-449e-8cc6-a2c5a10b8ee7 tempest-VolumeQuotaClassesTest-305704263 tempest-VolumeQuotaClassesTest-305704263-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '894a787687384f14be2c5cf412715100', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1c907159a9646cfab27300decf709d1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:30:48.374610 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef41e174-bd7e-449e-8cc6-a2c5a10b8ee7 tempest-VolumeQuotaClassesTest-305704263 tempest-VolumeQuotaClassesTest-305704263-project-admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:30:48.374610 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef41e174-bd7e-449e-8cc6-a2c5a10b8ee7 tempest-VolumeQuotaClassesTest-305704263 tempest-VolumeQuotaClassesTest-305704263-project-admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:30:48.375225 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef41e174-bd7e-449e-8cc6-a2c5a10b8ee7 tempest-VolumeQuotaClassesTest-305704263 tempest-VolumeQuotaClassesTest-305704263-project-admin] PATH_INFO: `/v3/domains` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:30:48.375225 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef41e174-bd7e-449e-8cc6-a2c5a10b8ee7 tempest-VolumeQuotaClassesTest-305704263 tempest-VolumeQuotaClassesTest-305704263-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:30:48.376146 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef41e174-bd7e-449e-8cc6-a2c5a10b8ee7 tempest-VolumeQuotaClassesTest-305704263 tempest-VolumeQuotaClassesTest-305704263-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:30:48.377904 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef41e174-bd7e-449e-8cc6-a2c5a10b8ee7 tempest-VolumeQuotaClassesTest-305704263 tempest-VolumeQuotaClassesTest-305704263-project-admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:30:48.395933 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1344/2663] 10.222.0.23 () {58 vars in 1152 bytes} [Thu May 19 12:30:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:30:48.460752 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff2d6c7-2288-4d88-a077-8a300d167da6 tempest-VolumeQuotaClassesTest-305704263 tempest-VolumeQuotaClassesTest-305704263-project-admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:30:48.472784 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff2d6c7-2288-4d88-a077-8a300d167da6 tempest-VolumeQuotaClassesTest-305704263 tempest-VolumeQuotaClassesTest-305704263-project-admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:30:48.477800 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9126847d-7dfb-4981-8529-488f48f7bdc1 tempest-VolumeQuotaClassesTest-305704263 tempest-VolumeQuotaClassesTest-305704263-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '894a787687384f14be2c5cf412715100', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1c907159a9646cfab27300decf709d1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:30:48.480050 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9126847d-7dfb-4981-8529-488f48f7bdc1 tempest-VolumeQuotaClassesTest-305704263 tempest-VolumeQuotaClassesTest-305704263-project-admin] REQUEST_METHOD: `DELETE` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:30:48.480050 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9126847d-7dfb-4981-8529-488f48f7bdc1 tempest-VolumeQuotaClassesTest-305704263 tempest-VolumeQuotaClassesTest-305704263-project-admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:30:48.480050 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9126847d-7dfb-4981-8529-488f48f7bdc1 tempest-VolumeQuotaClassesTest-305704263 tempest-VolumeQuotaClassesTest-305704263-project-admin] PATH_INFO: `/v3/projects/58e951fcf3e843b999c8ac83c78c5fe8` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:30:48.481419 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9126847d-7dfb-4981-8529-488f48f7bdc1 tempest-VolumeQuotaClassesTest-305704263 tempest-VolumeQuotaClassesTest-305704263-project-admin] RBAC: Authorizing `identity:delete_project(project_id=58e951fcf3e843b999c8ac83c78c5fe8)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:30:48.483459 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9126847d-7dfb-4981-8529-488f48f7bdc1 tempest-VolumeQuotaClassesTest-305704263 tempest-VolumeQuotaClassesTest-305704263-project-admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:30:48.648462 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-9126847d-7dfb-4981-8529-488f48f7bdc1 tempest-VolumeQuotaClassesTest-305704263 tempest-VolumeQuotaClassesTest-305704263-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '58e951fcf3e843b999c8ac83c78c5fe8', 'request_id': 'req-9126847d-7dfb-4981-8529-488f48f7bdc1'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:30:48.660757 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-9126847d-7dfb-4981-8529-488f48f7bdc1 tempest-VolumeQuotaClassesTest-305704263 tempest-VolumeQuotaClassesTest-305704263-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '58e951fcf3e843b999c8ac83c78c5fe8', 'request_id': 'req-9126847d-7dfb-4981-8529-488f48f7bdc1'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:30:48.751118 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-9126847d-7dfb-4981-8529-488f48f7bdc1 tempest-VolumeQuotaClassesTest-305704263 tempest-VolumeQuotaClassesTest-305704263-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '58e951fcf3e843b999c8ac83c78c5fe8', 'request_id': 'req-9126847d-7dfb-4981-8529-488f48f7bdc1'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:30:48.764700 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-9126847d-7dfb-4981-8529-488f48f7bdc1 tempest-VolumeQuotaClassesTest-305704263 tempest-VolumeQuotaClassesTest-305704263-project-admin] The token cache is being invalidate because project 58e951fcf3e843b999c8ac83c78c5fe8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80539) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 12:30:48.765174 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-9126847d-7dfb-4981-8529-488f48f7bdc1 tempest-VolumeQuotaClassesTest-305704263 tempest-VolumeQuotaClassesTest-305704263-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:30:48.765988 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-9126847d-7dfb-4981-8529-488f48f7bdc1 tempest-VolumeQuotaClassesTest-305704263 tempest-VolumeQuotaClassesTest-305704263-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:30:48.767755 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1320/2664] 10.222.0.23 () {58 vars in 1232 bytes} [Thu May 19 12:30:48 2022] DELETE /identity/v3/projects/58e951fcf3e843b999c8ac83c78c5fe8 => generated 0 bytes in 308 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:30:49.563568 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef41e174-bd7e-449e-8cc6-a2c5a10b8ee7 tempest-VolumeQuotaClassesTest-305704263 tempest-VolumeQuotaClassesTest-305704263-project-admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:30:49.565558 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-ef41e174-bd7e-449e-8cc6-a2c5a10b8ee7 tempest-VolumeQuotaClassesTest-305704263 tempest-VolumeQuotaClassesTest-305704263-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:30:49.651578 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef41e174-bd7e-449e-8cc6-a2c5a10b8ee7 tempest-VolumeQuotaClassesTest-305704263 tempest-VolumeQuotaClassesTest-305704263-project-admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:30:49.722673 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fa9980-ec8f-4b74-9fe6-55c256534c2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:30:49.724245 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fa9980-ec8f-4b74-9fe6-55c256534c2a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:30:49.724800 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fa9980-ec8f-4b74-9fe6-55c256534c2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:30:49.725176 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fa9980-ec8f-4b74-9fe6-55c256534c2a admin admin] PATH_INFO: `/v3/users/7c222658b1074dc4b6db18f9760a2099` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:30:49.726531 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56fa9980-ec8f-4b74-9fe6-55c256534c2a admin admin] RBAC: Authorizing `identity:delete_user(user_id=7c222658b1074dc4b6db18f9760a2099)` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:30:49.728615 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56fa9980-ec8f-4b74-9fe6-55c256534c2a admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:30:49.902285 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9126847d-7dfb-4981-8529-488f48f7bdc1 tempest-VolumeQuotaClassesTest-305704263 tempest-VolumeQuotaClassesTest-305704263-project-admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:30:49.904153 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-9126847d-7dfb-4981-8529-488f48f7bdc1 tempest-VolumeQuotaClassesTest-305704263 tempest-VolumeQuotaClassesTest-305704263-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:30:49.984714 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:30:49.992342 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9126847d-7dfb-4981-8529-488f48f7bdc1 tempest-VolumeQuotaClassesTest-305704263 tempest-VolumeQuotaClassesTest-305704263-project-admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:30:49.997069 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d196f4-b8b8-4843-8374-6af1c0452511 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8800254c0e784e42a791fb9abc274d79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd70d25bf69314820b8aaecb65576978a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:30:49.997598 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d196f4-b8b8-4843-8374-6af1c0452511 service cinder] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:30:49.997909 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d196f4-b8b8-4843-8374-6af1c0452511 service cinder] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:30:49.998165 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d196f4-b8b8-4843-8374-6af1c0452511 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:30:50.000361 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-d5d196f4-b8b8-4843-8374-6af1c0452511 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:30:50.000598 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-56fa9980-ec8f-4b74-9fe6-55c256534c2a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7c222658b1074dc4b6db18f9760a2099'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:30:50.032541 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-56fa9980-ec8f-4b74-9fe6-55c256534c2a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7c222658b1074dc4b6db18f9760a2099'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:30:50.033546 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-56fa9980-ec8f-4b74-9fe6-55c256534c2a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7c222658b1074dc4b6db18f9760a2099'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:30:50.037770 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:30:50.083158 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d196f4-b8b8-4843-8374-6af1c0452511 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:30:50.084722 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d196f4-b8b8-4843-8374-6af1c0452511 service cinder] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:30:50.097028 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1321/2665] 10.222.0.23 () {60 vars in 1365 bytes} [Thu May 19 12:30:49 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 12:30:50.145522 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-56fa9980-ec8f-4b74-9fe6-55c256534c2a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7c222658b1074dc4b6db18f9760a2099'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:30:50.159144 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-56fa9980-ec8f-4b74-9fe6-55c256534c2a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7c222658b1074dc4b6db18f9760a2099'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:30:50.169421 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1345/2666] 10.222.0.23 () {58 vars in 1223 bytes} [Thu May 19 12:30:49 2022] DELETE /identity/v3/users/7c222658b1074dc4b6db18f9760a2099 => generated 0 bytes in 606 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:30:50.175817 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f0bc96-723a-491a-bbb2-56c975f16d5a None None] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:30:50.176347 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f0bc96-723a-491a-bbb2-56c975f16d5a None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:30:50.176855 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f0bc96-723a-491a-bbb2-56c975f16d5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:30:50.207574 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.auth.core [None req-a6f0bc96-723a-491a-bbb2-56c975f16d5a None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:30:50.210084 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-a6f0bc96-723a-491a-bbb2-56c975f16d5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:30:50.303287 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1322/2667] 10.222.0.23 () {58 vars in 957 bytes} [Thu May 19 12:30:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:30:50.315100 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fa9980-ec8f-4b74-9fe6-55c256534c2a admin admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:30:50.317469 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-56fa9980-ec8f-4b74-9fe6-55c256534c2a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:30:50.404135 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fa9980-ec8f-4b74-9fe6-55c256534c2a admin admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:30:50.408222 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bde3a30-d8ca-42d2-811d-890300a6b82f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a00b3fa7497456895e5e0bf105c4fc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd70d25bf69314820b8aaecb65576978a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:30:50.408932 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bde3a30-d8ca-42d2-811d-890300a6b82f service neutron] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:30:50.409224 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bde3a30-d8ca-42d2-811d-890300a6b82f service neutron] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:30:50.409511 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bde3a30-d8ca-42d2-811d-890300a6b82f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:30:50.421536 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bde3a30-d8ca-42d2-811d-890300a6b82f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:30:50.423396 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bde3a30-d8ca-42d2-811d-890300a6b82f service neutron] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:30:50.435750 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1346/2668] 10.222.0.23 () {60 vars in 1365 bytes} [Thu May 19 12:30:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:30:50.967573 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f0bc96-723a-491a-bbb2-56c975f16d5a None None] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:30:50.969580 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-a6f0bc96-723a-491a-bbb2-56c975f16d5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:30:50.988973 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f0bc96-723a-491a-bbb2-56c975f16d5a None None] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:30:50.994431 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6455e2-8463-45d4-bb02-dfc31968a166 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:30:50.995678 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6455e2-8463-45d4-bb02-dfc31968a166 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:30:50.995940 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6455e2-8463-45d4-bb02-dfc31968a166 admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:30:50.996203 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6455e2-8463-45d4-bb02-dfc31968a166 admin admin] PATH_INFO: `/v3/projects/b4fd0bea1cd04935862d0f74cf5be84c` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:30:50.997788 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6455e2-8463-45d4-bb02-dfc31968a166 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b4fd0bea1cd04935862d0f74cf5be84c)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:30:50.999768 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6455e2-8463-45d4-bb02-dfc31968a166 admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:30:51.151043 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bde3a30-d8ca-42d2-811d-890300a6b82f service neutron] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:30:51.165664 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bde3a30-d8ca-42d2-811d-890300a6b82f service neutron] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:30:51.174097 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-7a6455e2-8463-45d4-bb02-dfc31968a166 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b4fd0bea1cd04935862d0f74cf5be84c', 'request_id': 'req-7a6455e2-8463-45d4-bb02-dfc31968a166'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:30:51.182782 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-7a6455e2-8463-45d4-bb02-dfc31968a166 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b4fd0bea1cd04935862d0f74cf5be84c', 'request_id': 'req-7a6455e2-8463-45d4-bb02-dfc31968a166'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:30:51.236257 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b369c39-b912-4790-b9ac-56d2afc8a025 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a00b3fa7497456895e5e0bf105c4fc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd70d25bf69314820b8aaecb65576978a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:30:51.236938 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b369c39-b912-4790-b9ac-56d2afc8a025 service neutron] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:30:51.237482 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b369c39-b912-4790-b9ac-56d2afc8a025 service neutron] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:30:51.237772 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b369c39-b912-4790-b9ac-56d2afc8a025 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:30:51.239864 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-3b369c39-b912-4790-b9ac-56d2afc8a025 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:30:51.306281 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-7a6455e2-8463-45d4-bb02-dfc31968a166 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b4fd0bea1cd04935862d0f74cf5be84c', 'request_id': 'req-7a6455e2-8463-45d4-bb02-dfc31968a166'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:30:51.325854 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-7a6455e2-8463-45d4-bb02-dfc31968a166 admin admin] The token cache is being invalidate because project b4fd0bea1cd04935862d0f74cf5be84c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80539) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 12:30:51.326317 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-7a6455e2-8463-45d4-bb02-dfc31968a166 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:30:51.327525 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-7a6455e2-8463-45d4-bb02-dfc31968a166 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:30:51.328331 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b369c39-b912-4790-b9ac-56d2afc8a025 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:30:51.330055 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1323/2669] 10.222.0.23 () {58 vars in 1232 bytes} [Thu May 19 12:30:50 2022] DELETE /identity/v3/projects/b4fd0bea1cd04935862d0f74cf5be84c => generated 0 bytes in 364 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:30:51.332574 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b369c39-b912-4790-b9ac-56d2afc8a025 service neutron] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:30:51.335088 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6455e2-8463-45d4-bb02-dfc31968a166 admin admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:30:51.337616 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-7a6455e2-8463-45d4-bb02-dfc31968a166 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:30:51.351603 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1347/2670] 10.222.0.23 () {60 vars in 1365 bytes} [Thu May 19 12:30:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 202 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:30:51.358754 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6455e2-8463-45d4-bb02-dfc31968a166 admin admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:30:51.365427 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6b1626-e44c-4be4-b3af-543b41b33f72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:30:51.366730 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6b1626-e44c-4be4-b3af-543b41b33f72 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:30:51.367095 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6b1626-e44c-4be4-b3af-543b41b33f72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:30:51.368031 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6b1626-e44c-4be4-b3af-543b41b33f72 admin admin] PATH_INFO: `/v3/users/894a787687384f14be2c5cf412715100` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:30:51.370070 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6b1626-e44c-4be4-b3af-543b41b33f72 admin admin] RBAC: Authorizing `identity:delete_user(user_id=894a787687384f14be2c5cf412715100)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:30:51.372721 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6b1626-e44c-4be4-b3af-543b41b33f72 admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:30:51.404484 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:30:51.697073 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-3c6b1626-e44c-4be4-b3af-543b41b33f72 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '894a787687384f14be2c5cf412715100'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:30:51.717897 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-3c6b1626-e44c-4be4-b3af-543b41b33f72 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '894a787687384f14be2c5cf412715100'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:30:51.718935 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-3c6b1626-e44c-4be4-b3af-543b41b33f72 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '894a787687384f14be2c5cf412715100'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:30:51.881621 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-3c6b1626-e44c-4be4-b3af-543b41b33f72 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '894a787687384f14be2c5cf412715100'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:30:51.896814 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-3c6b1626-e44c-4be4-b3af-543b41b33f72 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '894a787687384f14be2c5cf412715100'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:30:51.911331 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1324/2671] 10.222.0.23 () {58 vars in 1223 bytes} [Thu May 19 12:30:51 2022] DELETE /identity/v3/users/894a787687384f14be2c5cf412715100 => generated 0 bytes in 574 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 12:30:52.495395 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b369c39-b912-4790-b9ac-56d2afc8a025 service neutron] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:30:52.497852 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-3b369c39-b912-4790-b9ac-56d2afc8a025 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:30:52.580521 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:DELETE=1,keystone:SELECT=99 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:30:52.615158 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b369c39-b912-4790-b9ac-56d2afc8a025 service neutron] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:30:52.720451 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c5c167-6ec0-4a7d-89ab-bbaaaa9bd86b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8800254c0e784e42a791fb9abc274d79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd70d25bf69314820b8aaecb65576978a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:30:52.721903 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c5c167-6ec0-4a7d-89ab-bbaaaa9bd86b service cinder] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:30:52.722373 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c5c167-6ec0-4a7d-89ab-bbaaaa9bd86b service cinder] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:30:52.722865 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c5c167-6ec0-4a7d-89ab-bbaaaa9bd86b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:30:52.726542 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-04c5c167-6ec0-4a7d-89ab-bbaaaa9bd86b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:30:52.795381 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6b1626-e44c-4be4-b3af-543b41b33f72 admin admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:30:52.795381 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-3c6b1626-e44c-4be4-b3af-543b41b33f72 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:30:52.852748 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c5c167-6ec0-4a7d-89ab-bbaaaa9bd86b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:30:52.857070 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c5c167-6ec0-4a7d-89ab-bbaaaa9bd86b service cinder] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:30:52.872655 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1348/2672] 10.222.0.23 () {60 vars in 1365 bytes} [Thu May 19 12:30:52 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:30:52.910422 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6b1626-e44c-4be4-b3af-543b41b33f72 admin admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:30:52.916621 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf75cc2-13ee-495c-b972-5590a52a0511 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:30:52.917629 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf75cc2-13ee-495c-b972-5590a52a0511 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:30:52.917909 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf75cc2-13ee-495c-b972-5590a52a0511 admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:30:52.918182 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf75cc2-13ee-495c-b972-5590a52a0511 admin admin] PATH_INFO: `/v3/projects/b1c907159a9646cfab27300decf709d1` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:30:52.920614 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf75cc2-13ee-495c-b972-5590a52a0511 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b1c907159a9646cfab27300decf709d1)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:30:52.923376 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf75cc2-13ee-495c-b972-5590a52a0511 admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:30:53.120135 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-baf75cc2-13ee-495c-b972-5590a52a0511 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b1c907159a9646cfab27300decf709d1', 'request_id': 'req-baf75cc2-13ee-495c-b972-5590a52a0511'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:30:53.128200 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-baf75cc2-13ee-495c-b972-5590a52a0511 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b1c907159a9646cfab27300decf709d1', 'request_id': 'req-baf75cc2-13ee-495c-b972-5590a52a0511'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:30:53.260320 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-baf75cc2-13ee-495c-b972-5590a52a0511 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b1c907159a9646cfab27300decf709d1', 'request_id': 'req-baf75cc2-13ee-495c-b972-5590a52a0511'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:30:53.273906 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-baf75cc2-13ee-495c-b972-5590a52a0511 admin admin] The token cache is being invalidate because project b1c907159a9646cfab27300decf709d1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80539) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 12:30:53.274394 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-baf75cc2-13ee-495c-b972-5590a52a0511 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:30:53.275929 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-baf75cc2-13ee-495c-b972-5590a52a0511 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:30:53.277838 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1325/2673] 10.222.0.23 () {58 vars in 1232 bytes} [Thu May 19 12:30:52 2022] DELETE /identity/v3/projects/b1c907159a9646cfab27300decf709d1 => generated 0 bytes in 492 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 12:30:53.283732 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5e33bf-3522-4c8a-a635-bdb03f0169fe None None] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:30:53.284183 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5e33bf-3522-4c8a-a635-bdb03f0169fe None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:30:53.284350 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5e33bf-3522-4c8a-a635-bdb03f0169fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:30:53.305680 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.auth.core [None req-cd5e33bf-3522-4c8a-a635-bdb03f0169fe None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:30:53.307643 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-cd5e33bf-3522-4c8a-a635-bdb03f0169fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:30:53.405953 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1349/2674] 10.222.0.23 () {58 vars in 957 bytes} [Thu May 19 12:30:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:30:53.419657 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bce3366-c909-41ac-9825-89f0cd511dc5 None None] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:30:53.420034 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bce3366-c909-41ac-9825-89f0cd511dc5 None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:30:53.420250 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bce3366-c909-41ac-9825-89f0cd511dc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:30:53.441857 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.auth.core [None req-7bce3366-c909-41ac-9825-89f0cd511dc5 None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:30:53.443168 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-7bce3366-c909-41ac-9825-89f0cd511dc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:30:53.453430 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1326/2675] 10.222.0.23 () {58 vars in 957 bytes} [Thu May 19 12:30:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:30:53.458757 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5e33bf-3522-4c8a-a635-bdb03f0169fe None None] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:30:53.476330 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5e33bf-3522-4c8a-a635-bdb03f0169fe None None] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:30:53.482639 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb8d97b-172f-4952-b94c-013433ec6f6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:30:53.483337 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb8d97b-172f-4952-b94c-013433ec6f6f admin admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:30:53.483627 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb8d97b-172f-4952-b94c-013433ec6f6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:30:53.483951 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb8d97b-172f-4952-b94c-013433ec6f6f admin admin] PATH_INFO: `/v3/domains` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:30:53.484707 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb8d97b-172f-4952-b94c-013433ec6f6f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:30:53.485020 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb8d97b-172f-4952-b94c-013433ec6f6f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:30:53.486716 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb8d97b-172f-4952-b94c-013433ec6f6f admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:30:53.503445 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1350/2676] 10.222.0.23 () {58 vars in 1152 bytes} [Thu May 19 12:30:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:30:53.508115 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bce3366-c909-41ac-9825-89f0cd511dc5 None None] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:30:53.519583 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bce3366-c909-41ac-9825-89f0cd511dc5 None None] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:30:53.525020 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c84696-aea8-46dd-b0b2-8dedf17a9c77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:30:53.529688 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c84696-aea8-46dd-b0b2-8dedf17a9c77 admin admin] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:30:53.529688 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c84696-aea8-46dd-b0b2-8dedf17a9c77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:30:53.529688 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c84696-aea8-46dd-b0b2-8dedf17a9c77 admin admin] PATH_INFO: `/v3/projects` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:30:53.529688 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c84696-aea8-46dd-b0b2-8dedf17a9c77 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:30:53.530386 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c84696-aea8-46dd-b0b2-8dedf17a9c77 admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:30:53.638328 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:DELETE=15,keystone:SELECT=82,keystone:INSERT=3 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:30:53.647381 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1327/2677] 10.222.0.23 () {60 vars in 1152 bytes} [Thu May 19 12:30:53 2022] POST /identity/v3/projects => generated 407 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 12:30:53.653889 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb8d97b-172f-4952-b94c-013433ec6f6f admin admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:30:53.665342 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb8d97b-172f-4952-b94c-013433ec6f6f admin admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:30:53.709697 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:30:53.751927 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e938bd3-f3c8-415b-9797-321877a30f1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:30:53.752829 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e938bd3-f3c8-415b-9797-321877a30f1a admin admin] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:30:53.753027 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e938bd3-f3c8-415b-9797-321877a30f1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:30:53.753322 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e938bd3-f3c8-415b-9797-321877a30f1a admin admin] PATH_INFO: `/v3/users` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:30:53.754017 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e938bd3-f3c8-415b-9797-321877a30f1a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:30:53.756178 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e938bd3-f3c8-415b-9797-321877a30f1a admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:30:53.878895 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1351/2678] 10.222.0.23 () {60 vars in 1143 bytes} [Thu May 19 12:30:53 2022] POST /identity/v3/users => generated 304 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 12:30:53.886018 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c84696-aea8-46dd-b0b2-8dedf17a9c77 admin admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:30:53.912108 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c84696-aea8-46dd-b0b2-8dedf17a9c77 admin admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:30:53.926424 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5dd0e0-6d99-41ce-9d80-761bf2eabb4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:30:53.929714 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5dd0e0-6d99-41ce-9d80-761bf2eabb4b admin admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:30:53.930683 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5dd0e0-6d99-41ce-9d80-761bf2eabb4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:30:53.932040 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5dd0e0-6d99-41ce-9d80-761bf2eabb4b admin admin] PATH_INFO: `/v3/roles` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:30:53.934912 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5dd0e0-6d99-41ce-9d80-761bf2eabb4b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:30:53.939108 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5dd0e0-6d99-41ce-9d80-761bf2eabb4b admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:30:53.975840 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1328/2679] 10.222.0.23 () {58 vars in 1121 bytes} [Thu May 19 12:30:53 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 19 12:30:53.979254 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e938bd3-f3c8-415b-9797-321877a30f1a admin admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:30:53.998606 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e938bd3-f3c8-415b-9797-321877a30f1a admin admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:30:54.008655 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2320522a-6011-46fe-b8ec-f6084d418710 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:30:54.009563 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2320522a-6011-46fe-b8ec-f6084d418710 admin admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:30:54.010814 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2320522a-6011-46fe-b8ec-f6084d418710 admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:30:54.011469 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2320522a-6011-46fe-b8ec-f6084d418710 admin admin] PATH_INFO: `/v3/roles` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:30:54.013131 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2320522a-6011-46fe-b8ec-f6084d418710 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:30:54.016656 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2320522a-6011-46fe-b8ec-f6084d418710 admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:30:54.035195 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1352/2680] 10.222.0.23 () {58 vars in 1121 bytes} [Thu May 19 12:30:53 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 12:30:54.042637 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5dd0e0-6d99-41ce-9d80-761bf2eabb4b admin admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:30:54.061902 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5dd0e0-6d99-41ce-9d80-761bf2eabb4b admin admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:30:54.070227 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0580098-64bd-4070-9408-0868609f677b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:30:54.071894 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0580098-64bd-4070-9408-0868609f677b admin admin] REQUEST_METHOD: `PUT` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:30:54.072863 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0580098-64bd-4070-9408-0868609f677b admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:30:54.073169 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0580098-64bd-4070-9408-0868609f677b admin admin] PATH_INFO: `/v3/projects/b96f24f462014ae6a3dbf90008e7579f/users/0f9a99fbce9b4de08cd1662f5ddeaedc/roles/259eb400cdb4439db72d14456ae9cd08` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:30:54.097625 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0580098-64bd-4070-9408-0868609f677b admin admin] RBAC: Authorizing `identity:create_grant(project_id=b96f24f462014ae6a3dbf90008e7579f, user_id=0f9a99fbce9b4de08cd1662f5ddeaedc, role_id=259eb400cdb4439db72d14456ae9cd08)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:30:54.101494 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0580098-64bd-4070-9408-0868609f677b admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:30:54.183204 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1329/2681] 10.222.0.23 () {60 vars in 1482 bytes} [Thu May 19 12:30:54 2022] PUT /identity/v3/projects/b96f24f462014ae6a3dbf90008e7579f/users/0f9a99fbce9b4de08cd1662f5ddeaedc/roles/259eb400cdb4439db72d14456ae9cd08 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:30:54.196924 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f6ad22-a329-449a-a0da-06fe6356ccd7 None None] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:30:54.197704 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f6ad22-a329-449a-a0da-06fe6356ccd7 None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:30:54.198019 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f6ad22-a329-449a-a0da-06fe6356ccd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:30:54.258854 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.auth.core [None req-19f6ad22-a329-449a-a0da-06fe6356ccd7 None None] MFA Rules not processed for user `0f9a99fbce9b4de08cd1662f5ddeaedc`. Rule list: `[]` (Enabled: `True`). {{(pid=80540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:30:54.262676 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-19f6ad22-a329-449a-a0da-06fe6356ccd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:30:54.443339 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1353/2682] 10.222.0.23 () {58 vars in 957 bytes} [Thu May 19 12:30:54 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 249 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:30:54.450557 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0580098-64bd-4070-9408-0868609f677b admin admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:30:54.622042 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0580098-64bd-4070-9408-0868609f677b admin admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:30:54.632482 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbed15a4-afa9-48e1-bdb7-14cb7a1878c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:30:54.633658 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbed15a4-afa9-48e1-bdb7-14cb7a1878c0 admin admin] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:30:54.634160 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbed15a4-afa9-48e1-bdb7-14cb7a1878c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:30:54.634965 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbed15a4-afa9-48e1-bdb7-14cb7a1878c0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:30:54.637117 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbed15a4-afa9-48e1-bdb7-14cb7a1878c0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:30:54.641136 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbed15a4-afa9-48e1-bdb7-14cb7a1878c0 admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:30:54.768447 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1330/2683] 10.222.0.23 () {60 vars in 1152 bytes} [Thu May 19 12:30:54 2022] POST /identity/v3/projects => generated 407 bytes in 319 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 12:30:54.773350 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f6ad22-a329-449a-a0da-06fe6356ccd7 None None] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:30:54.802173 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f6ad22-a329-449a-a0da-06fe6356ccd7 None None] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:30:54.929033 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14f4e2d-21ef-47fb-9c97-32149541f9f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:30:54.929957 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14f4e2d-21ef-47fb-9c97-32149541f9f5 admin admin] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:30:54.930406 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14f4e2d-21ef-47fb-9c97-32149541f9f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:30:54.935855 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14f4e2d-21ef-47fb-9c97-32149541f9f5 admin admin] PATH_INFO: `/v3/users` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:30:54.935855 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14f4e2d-21ef-47fb-9c97-32149541f9f5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:30:54.935855 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14f4e2d-21ef-47fb-9c97-32149541f9f5 admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:30:55.082707 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1354/2684] 10.222.0.23 () {60 vars in 1143 bytes} [Thu May 19 12:30:54 2022] POST /identity/v3/users => generated 310 bytes in 310 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 12:30:55.089993 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbed15a4-afa9-48e1-bdb7-14cb7a1878c0 admin admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:30:55.107121 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbed15a4-afa9-48e1-bdb7-14cb7a1878c0 admin admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:30:55.118160 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a7be22-fbea-436b-89ce-f04963681c6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:30:55.119987 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a7be22-fbea-436b-89ce-f04963681c6d admin admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:30:55.120780 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a7be22-fbea-436b-89ce-f04963681c6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:30:55.122321 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a7be22-fbea-436b-89ce-f04963681c6d admin admin] PATH_INFO: `/v3/roles` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:30:55.123906 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a7be22-fbea-436b-89ce-f04963681c6d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:30:55.129431 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a7be22-fbea-436b-89ce-f04963681c6d admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:30:55.168679 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1331/2685] 10.222.0.23 () {58 vars in 1121 bytes} [Thu May 19 12:30:55 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 12:30:55.170654 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14f4e2d-21ef-47fb-9c97-32149541f9f5 admin admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:30:55.201864 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14f4e2d-21ef-47fb-9c97-32149541f9f5 admin admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:30:55.227749 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784c8f32-edd1-4a75-8277-15176bdb50c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:30:55.227749 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784c8f32-edd1-4a75-8277-15176bdb50c2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:30:55.227749 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784c8f32-edd1-4a75-8277-15176bdb50c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:30:55.227749 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784c8f32-edd1-4a75-8277-15176bdb50c2 admin admin] PATH_INFO: `/v3/projects/943f983348f54f508101cb946a8fd2f1/users/af93cbdeee3042078fa7782f6a6d1e63/roles/9db5671c65d3414ea2682afa2ccf1403` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:30:55.273960 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784c8f32-edd1-4a75-8277-15176bdb50c2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=943f983348f54f508101cb946a8fd2f1, user_id=af93cbdeee3042078fa7782f6a6d1e63, role_id=9db5671c65d3414ea2682afa2ccf1403)` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:30:55.281739 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784c8f32-edd1-4a75-8277-15176bdb50c2 admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:30:55.422259 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1355/2686] 10.222.0.23 () {60 vars in 1482 bytes} [Thu May 19 12:30:55 2022] PUT /identity/v3/projects/943f983348f54f508101cb946a8fd2f1/users/af93cbdeee3042078fa7782f6a6d1e63/roles/9db5671c65d3414ea2682afa2ccf1403 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:30:55.427263 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a7be22-fbea-436b-89ce-f04963681c6d admin admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:30:55.440807 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a7be22-fbea-436b-89ce-f04963681c6d admin admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:30:55.519291 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebec11a-f56e-476e-b8bc-0aeb2853bb0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:30:55.520307 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebec11a-f56e-476e-b8bc-0aeb2853bb0e admin admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:30:55.520940 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebec11a-f56e-476e-b8bc-0aeb2853bb0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:30:55.521428 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebec11a-f56e-476e-b8bc-0aeb2853bb0e admin admin] PATH_INFO: `/v3/roles` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:30:55.522866 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ebec11a-f56e-476e-b8bc-0aeb2853bb0e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:30:55.526220 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ebec11a-f56e-476e-b8bc-0aeb2853bb0e admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:30:55.542955 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1332/2687] 10.222.0.23 () {58 vars in 1121 bytes} [Thu May 19 12:30:55 2022] GET /identity/v3/roles => generated 1420 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 12:30:55.548322 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784c8f32-edd1-4a75-8277-15176bdb50c2 admin admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:30:55.555940 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:30:55.560567 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784c8f32-edd1-4a75-8277-15176bdb50c2 admin admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:30:55.573521 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c9a412-ad52-4a8a-9322-52a7350750ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:30:55.574646 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c9a412-ad52-4a8a-9322-52a7350750ac admin admin] REQUEST_METHOD: `PUT` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:30:55.574939 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c9a412-ad52-4a8a-9322-52a7350750ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:30:55.575253 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c9a412-ad52-4a8a-9322-52a7350750ac admin admin] PATH_INFO: `/v3/projects/943f983348f54f508101cb946a8fd2f1/users/af93cbdeee3042078fa7782f6a6d1e63/roles/9db5671c65d3414ea2682afa2ccf1403` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:30:55.577773 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c9a412-ad52-4a8a-9322-52a7350750ac admin admin] RBAC: Authorizing `identity:create_grant(project_id=943f983348f54f508101cb946a8fd2f1, user_id=af93cbdeee3042078fa7782f6a6d1e63, role_id=9db5671c65d3414ea2682afa2ccf1403)` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:30:55.581276 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c9a412-ad52-4a8a-9322-52a7350750ac admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:30:55.598908 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1356/2688] 10.222.0.23 () {60 vars in 1482 bytes} [Thu May 19 12:30:55 2022] PUT /identity/v3/projects/943f983348f54f508101cb946a8fd2f1/users/af93cbdeee3042078fa7782f6a6d1e63/roles/9db5671c65d3414ea2682afa2ccf1403 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:30:55.613447 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fa6029-4ea5-43f4-8774-81ca17c86765 None None] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:30:55.613758 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fa6029-4ea5-43f4-8774-81ca17c86765 None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:30:55.614146 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fa6029-4ea5-43f4-8774-81ca17c86765 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:30:55.648324 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.auth.core [None req-57fa6029-4ea5-43f4-8774-81ca17c86765 None None] MFA Rules not processed for user `af93cbdeee3042078fa7782f6a6d1e63`. Rule list: `[]` (Enabled: `True`). {{(pid=80539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:30:55.649597 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-57fa6029-4ea5-43f4-8774-81ca17c86765 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:30:55.675207 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:30:55.774945 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1333/2689] 10.222.0.23 () {58 vars in 957 bytes} [Thu May 19 12:30:55 2022] POST /identity/v3/auth/tokens => generated 3341 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 12:30:55.804428 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c9a412-ad52-4a8a-9322-52a7350750ac admin admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:30:55.808737 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-d4c9a412-ad52-4a8a-9322-52a7350750ac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:30:55.963382 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c9a412-ad52-4a8a-9322-52a7350750ac admin admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:30:55.968596 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e6a6ef-eece-4005-a2e6-82eb47d8505e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8800254c0e784e42a791fb9abc274d79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd70d25bf69314820b8aaecb65576978a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:30:55.969497 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e6a6ef-eece-4005-a2e6-82eb47d8505e service cinder] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:30:55.969921 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e6a6ef-eece-4005-a2e6-82eb47d8505e service cinder] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:30:55.970302 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e6a6ef-eece-4005-a2e6-82eb47d8505e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:30:55.992222 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e6a6ef-eece-4005-a2e6-82eb47d8505e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:30:55.997304 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e6a6ef-eece-4005-a2e6-82eb47d8505e service cinder] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:30:56.006548 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1357/2690] 10.222.0.23 () {60 vars in 1365 bytes} [Thu May 19 12:30:55 2022] GET /identity/v3/auth/tokens => generated 3341 bytes in 203 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 12:30:56.599071 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fa6029-4ea5-43f4-8774-81ca17c86765 None None] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:30:56.613386 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fa6029-4ea5-43f4-8774-81ca17c86765 None None] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:30:56.617791 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051a326a-383d-4b2f-a2f2-e345e43b9a5b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8800254c0e784e42a791fb9abc274d79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd70d25bf69314820b8aaecb65576978a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:30:56.622292 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051a326a-383d-4b2f-a2f2-e345e43b9a5b service cinder] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:30:56.622292 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051a326a-383d-4b2f-a2f2-e345e43b9a5b service cinder] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:30:56.622292 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051a326a-383d-4b2f-a2f2-e345e43b9a5b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:30:56.711589 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-051a326a-383d-4b2f-a2f2-e345e43b9a5b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:30:56.713220 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-051a326a-383d-4b2f-a2f2-e345e43b9a5b service cinder] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:30:56.724527 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1334/2691] 10.222.0.23 () {60 vars in 1365 bytes} [Thu May 19 12:30:56 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 12:30:57.054983 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e6a6ef-eece-4005-a2e6-82eb47d8505e service cinder] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:30:57.054983 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-86e6a6ef-eece-4005-a2e6-82eb47d8505e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:30:57.146335 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e6a6ef-eece-4005-a2e6-82eb47d8505e service cinder] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:30:57.154247 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97aa199-df56-4097-b6c7-3e00af40d980 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:30:57.155337 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97aa199-df56-4097-b6c7-3e00af40d980 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:30:57.155735 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97aa199-df56-4097-b6c7-3e00af40d980 admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:30:57.156230 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97aa199-df56-4097-b6c7-3e00af40d980 admin admin] PATH_INFO: `/v3/users/3cdc4cc4d83b4495aa44de70147a001c` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:30:57.157786 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97aa199-df56-4097-b6c7-3e00af40d980 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3cdc4cc4d83b4495aa44de70147a001c)` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:30:57.160513 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97aa199-df56-4097-b6c7-3e00af40d980 admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:30:57.401688 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-f97aa199-df56-4097-b6c7-3e00af40d980 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3cdc4cc4d83b4495aa44de70147a001c'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:30:57.426452 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-f97aa199-df56-4097-b6c7-3e00af40d980 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3cdc4cc4d83b4495aa44de70147a001c'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:30:57.427654 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-f97aa199-df56-4097-b6c7-3e00af40d980 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3cdc4cc4d83b4495aa44de70147a001c'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:30:57.586049 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-f97aa199-df56-4097-b6c7-3e00af40d980 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3cdc4cc4d83b4495aa44de70147a001c'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:30:57.599201 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-f97aa199-df56-4097-b6c7-3e00af40d980 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3cdc4cc4d83b4495aa44de70147a001c'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:30:57.611132 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1358/2692] 10.222.0.23 () {58 vars in 1223 bytes} [Thu May 19 12:30:57 2022] DELETE /identity/v3/users/3cdc4cc4d83b4495aa44de70147a001c => generated 0 bytes in 559 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 12:30:58.612280 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051a326a-383d-4b2f-a2f2-e345e43b9a5b service cinder] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:30:58.664515 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051a326a-383d-4b2f-a2f2-e345e43b9a5b service cinder] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:30:58.798839 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8313228-e5f7-4f39-8eb3-68a9afae5f21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:30:58.799717 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8313228-e5f7-4f39-8eb3-68a9afae5f21 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:30:58.800174 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8313228-e5f7-4f39-8eb3-68a9afae5f21 admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:30:58.800463 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8313228-e5f7-4f39-8eb3-68a9afae5f21 admin admin] PATH_INFO: `/v3/projects/1952e5f71bb245de912a1249fd7f6167` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:30:58.801830 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8313228-e5f7-4f39-8eb3-68a9afae5f21 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1952e5f71bb245de912a1249fd7f6167)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:30:58.803942 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8313228-e5f7-4f39-8eb3-68a9afae5f21 admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:30:59.048978 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-a8313228-e5f7-4f39-8eb3-68a9afae5f21 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1952e5f71bb245de912a1249fd7f6167', 'request_id': 'req-a8313228-e5f7-4f39-8eb3-68a9afae5f21'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:30:59.057039 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-a8313228-e5f7-4f39-8eb3-68a9afae5f21 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1952e5f71bb245de912a1249fd7f6167', 'request_id': 'req-a8313228-e5f7-4f39-8eb3-68a9afae5f21'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:30:59.062188 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:30:59.147359 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-a8313228-e5f7-4f39-8eb3-68a9afae5f21 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1952e5f71bb245de912a1249fd7f6167', 'request_id': 'req-a8313228-e5f7-4f39-8eb3-68a9afae5f21'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:30:59.164712 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-a8313228-e5f7-4f39-8eb3-68a9afae5f21 admin admin] The token cache is being invalidate because project 1952e5f71bb245de912a1249fd7f6167 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80539) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 12:30:59.165357 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-a8313228-e5f7-4f39-8eb3-68a9afae5f21 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:30:59.166894 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-a8313228-e5f7-4f39-8eb3-68a9afae5f21 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:30:59.169078 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1335/2693] 10.222.0.23 () {58 vars in 1232 bytes} [Thu May 19 12:30:58 2022] DELETE /identity/v3/projects/1952e5f71bb245de912a1249fd7f6167 => generated 0 bytes in 558 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:30:59.175467 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66034b51-6c68-4bd1-8cfb-43bcffa2a86d None None] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:30:59.177397 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66034b51-6c68-4bd1-8cfb-43bcffa2a86d None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:30:59.178330 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66034b51-6c68-4bd1-8cfb-43bcffa2a86d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:30:59.212518 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.auth.core [None req-66034b51-6c68-4bd1-8cfb-43bcffa2a86d None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:30:59.214717 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-66034b51-6c68-4bd1-8cfb-43bcffa2a86d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:30:59.230268 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:30:59.303367 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1359/2694] 10.222.0.23 () {58 vars in 957 bytes} [Thu May 19 12:30:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:30:59.312039 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e862f11-567c-4ca0-943a-6a0c04b1d48e None None] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:30:59.315980 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e862f11-567c-4ca0-943a-6a0c04b1d48e None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:30:59.315980 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e862f11-567c-4ca0-943a-6a0c04b1d48e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:30:59.333934 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.auth.core [None req-4e862f11-567c-4ca0-943a-6a0c04b1d48e None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:30:59.335396 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-4e862f11-567c-4ca0-943a-6a0c04b1d48e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:30:59.345120 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1336/2695] 10.222.0.23 () {58 vars in 957 bytes} [Thu May 19 12:30:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 12:30:59.350000 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66034b51-6c68-4bd1-8cfb-43bcffa2a86d None None] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:30:59.362296 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66034b51-6c68-4bd1-8cfb-43bcffa2a86d None None] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:30:59.367700 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c015c540-2a34-435d-834e-0a2f9a875d2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:30:59.368660 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c015c540-2a34-435d-834e-0a2f9a875d2f admin admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:30:59.369013 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c015c540-2a34-435d-834e-0a2f9a875d2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:30:59.369790 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c015c540-2a34-435d-834e-0a2f9a875d2f admin admin] PATH_INFO: `/v3/domains` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:30:59.372256 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c015c540-2a34-435d-834e-0a2f9a875d2f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:30:59.372589 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c015c540-2a34-435d-834e-0a2f9a875d2f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:30:59.374262 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c015c540-2a34-435d-834e-0a2f9a875d2f admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:30:59.391338 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1360/2696] 10.222.0.23 () {58 vars in 1152 bytes} [Thu May 19 12:30:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:30:59.396474 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e862f11-567c-4ca0-943a-6a0c04b1d48e None None] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:30:59.407377 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e862f11-567c-4ca0-943a-6a0c04b1d48e None None] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:30:59.412809 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185f5d03-4cd0-458c-88c0-ebabf51e275b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:30:59.413509 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185f5d03-4cd0-458c-88c0-ebabf51e275b admin admin] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:30:59.413816 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185f5d03-4cd0-458c-88c0-ebabf51e275b admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:30:59.414079 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185f5d03-4cd0-458c-88c0-ebabf51e275b admin admin] PATH_INFO: `/v3/projects` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:30:59.414880 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185f5d03-4cd0-458c-88c0-ebabf51e275b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:30:59.417665 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185f5d03-4cd0-458c-88c0-ebabf51e275b admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:30:59.532037 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1337/2697] 10.222.0.23 () {60 vars in 1152 bytes} [Thu May 19 12:30:59 2022] POST /identity/v3/projects => generated 363 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 12:30:59.533695 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c015c540-2a34-435d-834e-0a2f9a875d2f admin admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:30:59.549137 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c015c540-2a34-435d-834e-0a2f9a875d2f admin admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:30:59.642549 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2de1b7-4e41-4c64-a711-c5951c9e83b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:30:59.643234 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2de1b7-4e41-4c64-a711-c5951c9e83b0 admin admin] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:30:59.643522 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2de1b7-4e41-4c64-a711-c5951c9e83b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:30:59.643833 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2de1b7-4e41-4c64-a711-c5951c9e83b0 admin admin] PATH_INFO: `/v3/users` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:30:59.644530 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2de1b7-4e41-4c64-a711-c5951c9e83b0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:30:59.646479 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2de1b7-4e41-4c64-a711-c5951c9e83b0 admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:30:59.771896 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1361/2698] 10.222.0.23 () {60 vars in 1143 bytes} [Thu May 19 12:30:59 2022] POST /identity/v3/users => generated 282 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 12:30:59.777242 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185f5d03-4cd0-458c-88c0-ebabf51e275b admin admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:30:59.790577 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185f5d03-4cd0-458c-88c0-ebabf51e275b admin admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:30:59.797101 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9873116-f2a3-4c57-a420-2c8a5e65f120 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:30:59.797959 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9873116-f2a3-4c57-a420-2c8a5e65f120 admin admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:30:59.798067 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9873116-f2a3-4c57-a420-2c8a5e65f120 admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:30:59.798344 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9873116-f2a3-4c57-a420-2c8a5e65f120 admin admin] PATH_INFO: `/v3/roles` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:30:59.799157 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9873116-f2a3-4c57-a420-2c8a5e65f120 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:30:59.801944 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9873116-f2a3-4c57-a420-2c8a5e65f120 admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:30:59.814524 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1338/2699] 10.222.0.23 () {58 vars in 1121 bytes} [Thu May 19 12:30:59 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 12:30:59.821290 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2de1b7-4e41-4c64-a711-c5951c9e83b0 admin admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:30:59.836649 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2de1b7-4e41-4c64-a711-c5951c9e83b0 admin admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:30:59.844412 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca6d6b4-ae57-464f-8577-6c1b199dbc0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:30:59.845577 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca6d6b4-ae57-464f-8577-6c1b199dbc0a admin admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:30:59.845976 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca6d6b4-ae57-464f-8577-6c1b199dbc0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:30:59.846397 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca6d6b4-ae57-464f-8577-6c1b199dbc0a admin admin] PATH_INFO: `/v3/roles` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:30:59.848614 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca6d6b4-ae57-464f-8577-6c1b199dbc0a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:30:59.850747 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca6d6b4-ae57-464f-8577-6c1b199dbc0a admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:30:59.869933 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1362/2700] 10.222.0.23 () {58 vars in 1121 bytes} [Thu May 19 12:30:59 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 19 12:30:59.871878 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9873116-f2a3-4c57-a420-2c8a5e65f120 admin admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:30:59.880816 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca6d6b4-ae57-464f-8577-6c1b199dbc0a admin admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:30:59.885428 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-6ca6d6b4-ae57-464f-8577-6c1b199dbc0a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:30:59.888894 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9873116-f2a3-4c57-a420-2c8a5e65f120 admin admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:30:59.895364 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e931a22-8993-4703-aa66-57351e8194b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:30:59.896630 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e931a22-8993-4703-aa66-57351e8194b0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:30:59.897068 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e931a22-8993-4703-aa66-57351e8194b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:30:59.897432 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e931a22-8993-4703-aa66-57351e8194b0 admin admin] PATH_INFO: `/v3/projects/1878428dc5ae445eb085c999050cea53/users/d3bd602bec704b01b66ab9bb6afd5ff5/roles/259eb400cdb4439db72d14456ae9cd08` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:30:59.904131 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca6d6b4-ae57-464f-8577-6c1b199dbc0a admin admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:30:59.912883 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641cc198-e907-4223-8aaa-f7acd06e4d35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:30:59.914554 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641cc198-e907-4223-8aaa-f7acd06e4d35 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:30:59.915180 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641cc198-e907-4223-8aaa-f7acd06e4d35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:30:59.915637 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641cc198-e907-4223-8aaa-f7acd06e4d35 admin admin] PATH_INFO: `/v3/users/ae74c2a7dd774a18816bc1be80de6d92` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:30:59.918640 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641cc198-e907-4223-8aaa-f7acd06e4d35 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ae74c2a7dd774a18816bc1be80de6d92)` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:30:59.918834 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e931a22-8993-4703-aa66-57351e8194b0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1878428dc5ae445eb085c999050cea53, user_id=d3bd602bec704b01b66ab9bb6afd5ff5, role_id=259eb400cdb4439db72d14456ae9cd08)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:30:59.921693 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e931a22-8993-4703-aa66-57351e8194b0 admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:30:59.922618 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641cc198-e907-4223-8aaa-f7acd06e4d35 admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:31:00.041896 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1339/2701] 10.222.0.23 () {60 vars in 1482 bytes} [Thu May 19 12:30:59 2022] PUT /identity/v3/projects/1878428dc5ae445eb085c999050cea53/users/d3bd602bec704b01b66ab9bb6afd5ff5/roles/259eb400cdb4439db72d14456ae9cd08 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 12:31:00.059009 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2813e810-dee4-4faa-b62e-b31aea06c996 None None] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:31:00.059467 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2813e810-dee4-4faa-b62e-b31aea06c996 None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:31:00.059967 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2813e810-dee4-4faa-b62e-b31aea06c996 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:31:00.116970 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.auth.core [None req-2813e810-dee4-4faa-b62e-b31aea06c996 None None] MFA Rules not processed for user `d3bd602bec704b01b66ab9bb6afd5ff5`. Rule list: `[]` (Enabled: `True`). {{(pid=80539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:31:00.120093 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-2813e810-dee4-4faa-b62e-b31aea06c996 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:31:00.209730 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-641cc198-e907-4223-8aaa-f7acd06e4d35 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ae74c2a7dd774a18816bc1be80de6d92'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:31:00.233433 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-641cc198-e907-4223-8aaa-f7acd06e4d35 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ae74c2a7dd774a18816bc1be80de6d92'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:31:00.234955 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-641cc198-e907-4223-8aaa-f7acd06e4d35 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ae74c2a7dd774a18816bc1be80de6d92'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:31:00.236335 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1340/2702] 10.222.0.23 () {58 vars in 957 bytes} [Thu May 19 12:31:00 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:31:00.241822 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2813e810-dee4-4faa-b62e-b31aea06c996 None None] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:31:00.318649 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2813e810-dee4-4faa-b62e-b31aea06c996 None None] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:31:00.324974 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1a5026-c1f3-488c-8ae0-9f04ff1feec6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:31:00.325752 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1a5026-c1f3-488c-8ae0-9f04ff1feec6 admin admin] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:31:00.326183 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1a5026-c1f3-488c-8ae0-9f04ff1feec6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:31:00.326290 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1a5026-c1f3-488c-8ae0-9f04ff1feec6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:31:00.327011 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1a5026-c1f3-488c-8ae0-9f04ff1feec6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:31:00.329166 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1a5026-c1f3-488c-8ae0-9f04ff1feec6 admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:31:00.341335 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-641cc198-e907-4223-8aaa-f7acd06e4d35 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ae74c2a7dd774a18816bc1be80de6d92'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:31:00.345152 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:31:00.352500 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-641cc198-e907-4223-8aaa-f7acd06e4d35 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ae74c2a7dd774a18816bc1be80de6d92'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:31:00.359591 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1363/2703] 10.222.0.23 () {58 vars in 1223 bytes} [Thu May 19 12:30:59 2022] DELETE /identity/v3/users/ae74c2a7dd774a18816bc1be80de6d92 => generated 0 bytes in 480 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 12:31:00.364452 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b9da47-1b9f-4893-a977-668b99d1fce7 None None] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:31:00.364742 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b9da47-1b9f-4893-a977-668b99d1fce7 None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:31:00.365023 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b9da47-1b9f-4893-a977-668b99d1fce7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:31:00.385909 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.auth.core [None req-c2b9da47-1b9f-4893-a977-668b99d1fce7 None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:31:00.388078 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-c2b9da47-1b9f-4893-a977-668b99d1fce7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:31:00.460150 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1364/2704] 10.222.0.23 () {58 vars in 957 bytes} [Thu May 19 12:31:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 12:31:00.471029 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b9da47-1b9f-4893-a977-668b99d1fce7 None None] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:31:00.471947 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:31:00.473566 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-c2b9da47-1b9f-4893-a977-668b99d1fce7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:31:00.483903 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1341/2705] 10.222.0.23 () {60 vars in 1152 bytes} [Thu May 19 12:31:00 2022] POST /identity/v3/projects => generated 363 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 12:31:00.485229 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1a5026-c1f3-488c-8ae0-9f04ff1feec6 admin admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:31:00.487271 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-aa1a5026-c1f3-488c-8ae0-9f04ff1feec6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:31:00.547065 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b9da47-1b9f-4893-a977-668b99d1fce7 None None] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:31:00.574964 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1a5026-c1f3-488c-8ae0-9f04ff1feec6 admin admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:31:00.582344 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0be6dd1-0b8d-41a5-a427-22da218d9103 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:31:00.583047 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0be6dd1-0b8d-41a5-a427-22da218d9103 admin admin] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:31:00.583381 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0be6dd1-0b8d-41a5-a427-22da218d9103 admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:31:00.583667 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0be6dd1-0b8d-41a5-a427-22da218d9103 admin admin] PATH_INFO: `/v3/users` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:31:00.584481 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0be6dd1-0b8d-41a5-a427-22da218d9103 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:31:00.586500 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0be6dd1-0b8d-41a5-a427-22da218d9103 admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:31:00.619478 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd7094d-944d-428c-9aa1-4f94a9801cc4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a00b3fa7497456895e5e0bf105c4fc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd70d25bf69314820b8aaecb65576978a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:31:00.620640 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd7094d-944d-428c-9aa1-4f94a9801cc4 service neutron] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:31:00.620937 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd7094d-944d-428c-9aa1-4f94a9801cc4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:31:00.621210 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd7094d-944d-428c-9aa1-4f94a9801cc4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:31:00.631356 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd7094d-944d-428c-9aa1-4f94a9801cc4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:31:00.632876 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd7094d-944d-428c-9aa1-4f94a9801cc4 service neutron] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:31:00.644532 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1365/2706] 10.222.0.23 () {60 vars in 1365 bytes} [Thu May 19 12:31:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 12:31:00.715888 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1342/2707] 10.222.0.23 () {60 vars in 1143 bytes} [Thu May 19 12:31:00 2022] POST /identity/v3/users => generated 288 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 12:31:00.717991 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd7094d-944d-428c-9aa1-4f94a9801cc4 service neutron] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:31:00.731708 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd7094d-944d-428c-9aa1-4f94a9801cc4 service neutron] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:31:00.737183 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99099d8c-447b-411e-8c47-ec261a5405f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:31:00.738143 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99099d8c-447b-411e-8c47-ec261a5405f8 admin admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:31:00.738434 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99099d8c-447b-411e-8c47-ec261a5405f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:31:00.738723 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99099d8c-447b-411e-8c47-ec261a5405f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:31:00.739559 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99099d8c-447b-411e-8c47-ec261a5405f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:31:00.741294 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99099d8c-447b-411e-8c47-ec261a5405f8 admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:31:00.754544 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1366/2708] 10.222.0.23 () {58 vars in 1121 bytes} [Thu May 19 12:31:00 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 12:31:00.761027 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0be6dd1-0b8d-41a5-a427-22da218d9103 admin admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:31:00.775719 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0be6dd1-0b8d-41a5-a427-22da218d9103 admin admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:31:00.782042 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9905e24-8b6d-4ebc-a5d9-9fe230c2245a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:31:00.783552 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9905e24-8b6d-4ebc-a5d9-9fe230c2245a admin admin] REQUEST_METHOD: `PUT` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:31:00.784027 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9905e24-8b6d-4ebc-a5d9-9fe230c2245a admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:31:00.784434 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9905e24-8b6d-4ebc-a5d9-9fe230c2245a admin admin] PATH_INFO: `/v3/projects/178730f5589f472da96cc280c8b80c9c/users/7d54049a8cf14872a6ec27687aaa5964/roles/9db5671c65d3414ea2682afa2ccf1403` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:31:00.808432 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9905e24-8b6d-4ebc-a5d9-9fe230c2245a admin admin] RBAC: Authorizing `identity:create_grant(project_id=178730f5589f472da96cc280c8b80c9c, user_id=7d54049a8cf14872a6ec27687aaa5964, role_id=9db5671c65d3414ea2682afa2ccf1403)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:31:00.811279 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9905e24-8b6d-4ebc-a5d9-9fe230c2245a admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:31:00.922733 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1343/2709] 10.222.0.23 () {60 vars in 1482 bytes} [Thu May 19 12:31:00 2022] PUT /identity/v3/projects/178730f5589f472da96cc280c8b80c9c/users/7d54049a8cf14872a6ec27687aaa5964/roles/9db5671c65d3414ea2682afa2ccf1403 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:31:00.935716 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99099d8c-447b-411e-8c47-ec261a5405f8 admin admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:31:00.959941 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99099d8c-447b-411e-8c47-ec261a5405f8 admin admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:31:01.028238 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:31:01.058326 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cce8d81-0876-45c2-aeae-c1f0e7c4a7d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:31:01.059215 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cce8d81-0876-45c2-aeae-c1f0e7c4a7d9 admin admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:31:01.059623 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cce8d81-0876-45c2-aeae-c1f0e7c4a7d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:31:01.060065 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cce8d81-0876-45c2-aeae-c1f0e7c4a7d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:31:01.060981 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cce8d81-0876-45c2-aeae-c1f0e7c4a7d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:31:01.064182 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cce8d81-0876-45c2-aeae-c1f0e7c4a7d9 admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:31:01.078216 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1367/2710] 10.222.0.23 () {58 vars in 1121 bytes} [Thu May 19 12:31:00 2022] GET /identity/v3/roles => generated 1420 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 12:31:01.082833 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9905e24-8b6d-4ebc-a5d9-9fe230c2245a admin admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:31:01.094742 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9905e24-8b6d-4ebc-a5d9-9fe230c2245a admin admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:31:01.099641 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704fdb87-9c9d-4b09-894d-7bba2cfb2424 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:31:01.100598 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704fdb87-9c9d-4b09-894d-7bba2cfb2424 admin admin] REQUEST_METHOD: `PUT` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:31:01.100887 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704fdb87-9c9d-4b09-894d-7bba2cfb2424 admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:31:01.101197 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704fdb87-9c9d-4b09-894d-7bba2cfb2424 admin admin] PATH_INFO: `/v3/projects/178730f5589f472da96cc280c8b80c9c/users/7d54049a8cf14872a6ec27687aaa5964/roles/9db5671c65d3414ea2682afa2ccf1403` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:31:01.104502 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-704fdb87-9c9d-4b09-894d-7bba2cfb2424 admin admin] RBAC: Authorizing `identity:create_grant(project_id=178730f5589f472da96cc280c8b80c9c, user_id=7d54049a8cf14872a6ec27687aaa5964, role_id=9db5671c65d3414ea2682afa2ccf1403)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:31:01.107257 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-704fdb87-9c9d-4b09-894d-7bba2cfb2424 admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:31:01.121190 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1344/2711] 10.222.0.23 () {60 vars in 1482 bytes} [Thu May 19 12:31:01 2022] PUT /identity/v3/projects/178730f5589f472da96cc280c8b80c9c/users/7d54049a8cf14872a6ec27687aaa5964/roles/9db5671c65d3414ea2682afa2ccf1403 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 12:31:01.141541 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695bfcba-83ea-4688-aaea-d14d7b1144ac None None] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:31:01.141865 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695bfcba-83ea-4688-aaea-d14d7b1144ac None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:31:01.142260 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695bfcba-83ea-4688-aaea-d14d7b1144ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:31:01.176648 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.auth.core [None req-695bfcba-83ea-4688-aaea-d14d7b1144ac None None] MFA Rules not processed for user `7d54049a8cf14872a6ec27687aaa5964`. Rule list: `[]` (Enabled: `True`). {{(pid=80540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:31:01.178156 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-695bfcba-83ea-4688-aaea-d14d7b1144ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:31:01.185961 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704fdb87-9c9d-4b09-894d-7bba2cfb2424 admin admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:31:01.189315 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-704fdb87-9c9d-4b09-894d-7bba2cfb2424 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:31:01.297612 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1368/2712] 10.222.0.23 () {58 vars in 957 bytes} [Thu May 19 12:31:01 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 12:31:01.314779 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695bfcba-83ea-4688-aaea-d14d7b1144ac None None] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:31:01.316350 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-695bfcba-83ea-4688-aaea-d14d7b1144ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:31:01.317520 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704fdb87-9c9d-4b09-894d-7bba2cfb2424 admin admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:31:01.323901 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea60d41f-63a2-4e12-9e72-6c445095651f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:31:01.325002 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea60d41f-63a2-4e12-9e72-6c445095651f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:31:01.325350 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea60d41f-63a2-4e12-9e72-6c445095651f admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:31:01.325706 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea60d41f-63a2-4e12-9e72-6c445095651f admin admin] PATH_INFO: `/v3/projects/faf9e920a9484c7f99e283ac8579267c` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:31:01.327655 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea60d41f-63a2-4e12-9e72-6c445095651f admin admin] RBAC: Authorizing `identity:delete_project(project_id=faf9e920a9484c7f99e283ac8579267c)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:31:01.331304 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea60d41f-63a2-4e12-9e72-6c445095651f admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:31:01.424290 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695bfcba-83ea-4688-aaea-d14d7b1144ac None None] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:31:01.431347 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acda998d-9382-4ec9-8ccf-01a97e6b3cbe service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8800254c0e784e42a791fb9abc274d79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd70d25bf69314820b8aaecb65576978a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:31:01.433179 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acda998d-9382-4ec9-8ccf-01a97e6b3cbe service cinder] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:31:01.433690 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acda998d-9382-4ec9-8ccf-01a97e6b3cbe service cinder] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:31:01.434329 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acda998d-9382-4ec9-8ccf-01a97e6b3cbe service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:31:01.452470 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acda998d-9382-4ec9-8ccf-01a97e6b3cbe service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:31:01.455544 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acda998d-9382-4ec9-8ccf-01a97e6b3cbe service cinder] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:31:01.475600 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-ea60d41f-63a2-4e12-9e72-6c445095651f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'faf9e920a9484c7f99e283ac8579267c', 'request_id': 'req-ea60d41f-63a2-4e12-9e72-6c445095651f'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:31:01.476371 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1369/2713] 10.222.0.23 () {60 vars in 1365 bytes} [Thu May 19 12:31:01 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 12:31:01.480211 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09112deb-2cca-4ae7-ae8d-5c952f4d98c9 None None] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:31:01.480543 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=91,keystone:INSERT=5,keystone:DELETE=4 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:31:01.480723 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09112deb-2cca-4ae7-ae8d-5c952f4d98c9 None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:31:01.480723 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09112deb-2cca-4ae7-ae8d-5c952f4d98c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:31:01.484747 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-ea60d41f-63a2-4e12-9e72-6c445095651f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'faf9e920a9484c7f99e283ac8579267c', 'request_id': 'req-ea60d41f-63a2-4e12-9e72-6c445095651f'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:31:01.508045 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.auth.core [None req-09112deb-2cca-4ae7-ae8d-5c952f4d98c9 None None] MFA Rules not processed for user `2d158b01a0a74de4bc0131b5c625efdf`. Rule list: `[]` (Enabled: `True`). {{(pid=80540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:31:01.509437 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-09112deb-2cca-4ae7-ae8d-5c952f4d98c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:31:01.579884 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1370/2714] 10.222.0.23 () {58 vars in 957 bytes} [Thu May 19 12:31:01 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:31:01.585771 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-ea60d41f-63a2-4e12-9e72-6c445095651f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'faf9e920a9484c7f99e283ac8579267c', 'request_id': 'req-ea60d41f-63a2-4e12-9e72-6c445095651f'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:31:01.600615 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-ea60d41f-63a2-4e12-9e72-6c445095651f admin admin] The token cache is being invalidate because project faf9e920a9484c7f99e283ac8579267c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80539) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 12:31:01.601200 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-ea60d41f-63a2-4e12-9e72-6c445095651f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:31:01.602101 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-ea60d41f-63a2-4e12-9e72-6c445095651f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:31:01.605750 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1345/2715] 10.222.0.23 () {58 vars in 1232 bytes} [Thu May 19 12:31:01 2022] DELETE /identity/v3/projects/faf9e920a9484c7f99e283ac8579267c => generated 0 bytes in 421 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:31:01.613579 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09112deb-2cca-4ae7-ae8d-5c952f4d98c9 None None] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:31:01.632450 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:31:01.706464 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09112deb-2cca-4ae7-ae8d-5c952f4d98c9 None None] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:31:01.713202 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11fe363-7e8a-4acd-b0e4-31a5fbce0139 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:31:01.714467 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11fe363-7e8a-4acd-b0e4-31a5fbce0139 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:31:01.714830 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11fe363-7e8a-4acd-b0e4-31a5fbce0139 admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:31:01.715229 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11fe363-7e8a-4acd-b0e4-31a5fbce0139 admin admin] PATH_INFO: `/v3/users/e48965a7840448b6b6c8524137f6edf0` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:31:01.716939 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11fe363-7e8a-4acd-b0e4-31a5fbce0139 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e48965a7840448b6b6c8524137f6edf0)` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:31:01.719219 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11fe363-7e8a-4acd-b0e4-31a5fbce0139 admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:31:01.755425 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea60d41f-63a2-4e12-9e72-6c445095651f admin admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:31:01.757151 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-ea60d41f-63a2-4e12-9e72-6c445095651f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:31:01.861624 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea60d41f-63a2-4e12-9e72-6c445095651f admin admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:31:01.869291 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42bdc06-d5b5-4401-b552-e53061a354b9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8800254c0e784e42a791fb9abc274d79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd70d25bf69314820b8aaecb65576978a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:31:01.870433 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42bdc06-d5b5-4401-b552-e53061a354b9 service cinder] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:31:01.870814 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42bdc06-d5b5-4401-b552-e53061a354b9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:31:01.871176 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42bdc06-d5b5-4401-b552-e53061a354b9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:31:01.874091 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-e42bdc06-d5b5-4401-b552-e53061a354b9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:31:01.963973 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42bdc06-d5b5-4401-b552-e53061a354b9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:31:01.965587 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42bdc06-d5b5-4401-b552-e53061a354b9 service cinder] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:31:01.978224 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1346/2716] 10.222.0.23 () {60 vars in 1365 bytes} [Thu May 19 12:31:01 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 223 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 12:31:02.043179 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-c11fe363-7e8a-4acd-b0e4-31a5fbce0139 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e48965a7840448b6b6c8524137f6edf0'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:31:02.057291 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-c11fe363-7e8a-4acd-b0e4-31a5fbce0139 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e48965a7840448b6b6c8524137f6edf0'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:31:02.058105 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-c11fe363-7e8a-4acd-b0e4-31a5fbce0139 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e48965a7840448b6b6c8524137f6edf0'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:31:02.089158 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42bdc06-d5b5-4401-b552-e53061a354b9 service cinder] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:31:02.090750 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-e42bdc06-d5b5-4401-b552-e53061a354b9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:31:02.159237 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42bdc06-d5b5-4401-b552-e53061a354b9 service cinder] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:31:02.163377 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af9ec8f-babf-4dc4-b8b2-1652c4cd0cc3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a00b3fa7497456895e5e0bf105c4fc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd70d25bf69314820b8aaecb65576978a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:31:02.164334 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af9ec8f-babf-4dc4-b8b2-1652c4cd0cc3 service neutron] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:31:02.164620 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af9ec8f-babf-4dc4-b8b2-1652c4cd0cc3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:31:02.164900 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af9ec8f-babf-4dc4-b8b2-1652c4cd0cc3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:31:02.168371 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-6af9ec8f-babf-4dc4-b8b2-1652c4cd0cc3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:31:02.179751 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af9ec8f-babf-4dc4-b8b2-1652c4cd0cc3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:31:02.181365 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af9ec8f-babf-4dc4-b8b2-1652c4cd0cc3 service neutron] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:31:02.192250 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1347/2717] 10.222.0.23 () {60 vars in 1365 bytes} [Thu May 19 12:31:02 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 12:31:02.209527 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-c11fe363-7e8a-4acd-b0e4-31a5fbce0139 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e48965a7840448b6b6c8524137f6edf0'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:31:02.216512 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-c11fe363-7e8a-4acd-b0e4-31a5fbce0139 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e48965a7840448b6b6c8524137f6edf0'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:31:02.226826 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1371/2718] 10.222.0.23 () {58 vars in 1223 bytes} [Thu May 19 12:31:01 2022] DELETE /identity/v3/users/e48965a7840448b6b6c8524137f6edf0 => generated 0 bytes in 614 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:31:02.824113 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af9ec8f-babf-4dc4-b8b2-1652c4cd0cc3 service neutron] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:31:02.825982 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-6af9ec8f-babf-4dc4-b8b2-1652c4cd0cc3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:31:02.842179 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af9ec8f-babf-4dc4-b8b2-1652c4cd0cc3 service neutron] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:31:02.888392 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:31:02.919110 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab92eb2-cedd-48ce-af9f-bac31d506b1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:31:02.921596 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab92eb2-cedd-48ce-af9f-bac31d506b1f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:31:02.921893 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab92eb2-cedd-48ce-af9f-bac31d506b1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:31:02.923231 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab92eb2-cedd-48ce-af9f-bac31d506b1f admin admin] PATH_INFO: `/v3/projects/fbc3a9ef12194870b924c6590b91fc1b` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:31:02.925602 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab92eb2-cedd-48ce-af9f-bac31d506b1f admin admin] RBAC: Authorizing `identity:delete_project(project_id=fbc3a9ef12194870b924c6590b91fc1b)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:31:02.931984 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab92eb2-cedd-48ce-af9f-bac31d506b1f admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:31:03.083029 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-0ab92eb2-cedd-48ce-af9f-bac31d506b1f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fbc3a9ef12194870b924c6590b91fc1b', 'request_id': 'req-0ab92eb2-cedd-48ce-af9f-bac31d506b1f'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:31:03.093301 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-0ab92eb2-cedd-48ce-af9f-bac31d506b1f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fbc3a9ef12194870b924c6590b91fc1b', 'request_id': 'req-0ab92eb2-cedd-48ce-af9f-bac31d506b1f'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:31:03.186405 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-0ab92eb2-cedd-48ce-af9f-bac31d506b1f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fbc3a9ef12194870b924c6590b91fc1b', 'request_id': 'req-0ab92eb2-cedd-48ce-af9f-bac31d506b1f'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:31:03.206000 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-0ab92eb2-cedd-48ce-af9f-bac31d506b1f admin admin] The token cache is being invalidate because project fbc3a9ef12194870b924c6590b91fc1b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80539) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 12:31:03.206514 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-0ab92eb2-cedd-48ce-af9f-bac31d506b1f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:31:03.207677 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-0ab92eb2-cedd-48ce-af9f-bac31d506b1f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:31:03.211697 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1348/2719] 10.222.0.23 () {58 vars in 1232 bytes} [Thu May 19 12:31:02 2022] DELETE /identity/v3/projects/fbc3a9ef12194870b924c6590b91fc1b => generated 0 bytes in 389 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:31:03.216676 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11fe363-7e8a-4acd-b0e4-31a5fbce0139 admin admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:31:03.218177 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-c11fe363-7e8a-4acd-b0e4-31a5fbce0139 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:31:03.239585 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11fe363-7e8a-4acd-b0e4-31a5fbce0139 admin admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:31:03.315340 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066208b4-918e-46d6-9809-cbbc7fa483ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:31:03.317354 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066208b4-918e-46d6-9809-cbbc7fa483ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:31:03.317878 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066208b4-918e-46d6-9809-cbbc7fa483ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:31:03.318397 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066208b4-918e-46d6-9809-cbbc7fa483ac admin admin] PATH_INFO: `/v3/users/8bc14ec9db0042e8a51a27d401bd7a76` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:31:03.320883 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066208b4-918e-46d6-9809-cbbc7fa483ac admin admin] RBAC: Authorizing `identity:delete_user(user_id=8bc14ec9db0042e8a51a27d401bd7a76)` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:31:03.325039 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066208b4-918e-46d6-9809-cbbc7fa483ac admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:31:03.551218 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:31:03.567056 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-066208b4-918e-46d6-9809-cbbc7fa483ac admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8bc14ec9db0042e8a51a27d401bd7a76'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:31:03.585722 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-066208b4-918e-46d6-9809-cbbc7fa483ac admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8bc14ec9db0042e8a51a27d401bd7a76'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:31:03.586760 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-066208b4-918e-46d6-9809-cbbc7fa483ac admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8bc14ec9db0042e8a51a27d401bd7a76'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:31:03.694994 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-066208b4-918e-46d6-9809-cbbc7fa483ac admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8bc14ec9db0042e8a51a27d401bd7a76'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:31:03.707959 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-066208b4-918e-46d6-9809-cbbc7fa483ac admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8bc14ec9db0042e8a51a27d401bd7a76'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:31:03.722137 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1372/2720] 10.222.0.23 () {58 vars in 1223 bytes} [Thu May 19 12:31:03 2022] DELETE /identity/v3/users/8bc14ec9db0042e8a51a27d401bd7a76 => generated 0 bytes in 502 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 12:31:04.361710 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab92eb2-cedd-48ce-af9f-bac31d506b1f admin admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:31:04.364694 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-0ab92eb2-cedd-48ce-af9f-bac31d506b1f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:31:04.386284 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab92eb2-cedd-48ce-af9f-bac31d506b1f admin admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:31:04.482035 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1a4183-3ac7-45ab-9b2a-c00b41c50e96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:31:04.483491 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1a4183-3ac7-45ab-9b2a-c00b41c50e96 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:31:04.484038 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1a4183-3ac7-45ab-9b2a-c00b41c50e96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:31:04.484551 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1a4183-3ac7-45ab-9b2a-c00b41c50e96 admin admin] PATH_INFO: `/v3/projects/9575ff3f86604308bc12b4b2cc6ff27a` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:31:04.486865 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1a4183-3ac7-45ab-9b2a-c00b41c50e96 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9575ff3f86604308bc12b4b2cc6ff27a)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:31:04.491441 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1a4183-3ac7-45ab-9b2a-c00b41c50e96 admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:31:04.617243 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-4e1a4183-3ac7-45ab-9b2a-c00b41c50e96 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9575ff3f86604308bc12b4b2cc6ff27a', 'request_id': 'req-4e1a4183-3ac7-45ab-9b2a-c00b41c50e96'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:31:04.624784 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-4e1a4183-3ac7-45ab-9b2a-c00b41c50e96 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9575ff3f86604308bc12b4b2cc6ff27a', 'request_id': 'req-4e1a4183-3ac7-45ab-9b2a-c00b41c50e96'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:31:04.734525 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-4e1a4183-3ac7-45ab-9b2a-c00b41c50e96 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9575ff3f86604308bc12b4b2cc6ff27a', 'request_id': 'req-4e1a4183-3ac7-45ab-9b2a-c00b41c50e96'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:31:04.747975 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:31:04.753593 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-4e1a4183-3ac7-45ab-9b2a-c00b41c50e96 admin admin] The token cache is being invalidate because project 9575ff3f86604308bc12b4b2cc6ff27a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80539) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 12:31:04.753964 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-4e1a4183-3ac7-45ab-9b2a-c00b41c50e96 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:31:04.754768 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-4e1a4183-3ac7-45ab-9b2a-c00b41c50e96 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:31:04.756672 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1349/2721] 10.222.0.23 () {58 vars in 1232 bytes} [Thu May 19 12:31:04 2022] DELETE /identity/v3/projects/9575ff3f86604308bc12b4b2cc6ff27a => generated 0 bytes in 396 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:31:04.762513 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf248cc-46f6-431d-a2b9-354e16c6c011 None None] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:31:04.763418 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf248cc-46f6-431d-a2b9-354e16c6c011 None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:31:04.763712 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf248cc-46f6-431d-a2b9-354e16c6c011 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:31:04.786783 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.auth.core [None req-6cf248cc-46f6-431d-a2b9-354e16c6c011 None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:31:04.788121 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-6cf248cc-46f6-431d-a2b9-354e16c6c011 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:31:04.894043 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1373/2722] 10.222.0.23 () {58 vars in 957 bytes} [Thu May 19 12:31:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 12:31:04.907152 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2eb1a6-3e75-41f0-9b56-2a672009a3c3 None None] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:31:04.907492 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2eb1a6-3e75-41f0-9b56-2a672009a3c3 None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:31:04.907760 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2eb1a6-3e75-41f0-9b56-2a672009a3c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:31:04.937805 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.auth.core [None req-1e2eb1a6-3e75-41f0-9b56-2a672009a3c3 None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:31:04.943417 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-1e2eb1a6-3e75-41f0-9b56-2a672009a3c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:31:04.957962 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1350/2723] 10.222.0.23 () {58 vars in 957 bytes} [Thu May 19 12:31:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:31:04.962693 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf248cc-46f6-431d-a2b9-354e16c6c011 None None] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:31:04.974045 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf248cc-46f6-431d-a2b9-354e16c6c011 None None] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:31:04.979606 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c13b9d-cbf3-4931-a6e8-c5fbed077b47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:31:04.985353 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c13b9d-cbf3-4931-a6e8-c5fbed077b47 admin admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:31:04.985353 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c13b9d-cbf3-4931-a6e8-c5fbed077b47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:31:04.985353 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c13b9d-cbf3-4931-a6e8-c5fbed077b47 admin admin] PATH_INFO: `/v3/domains` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:31:04.985353 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c13b9d-cbf3-4931-a6e8-c5fbed077b47 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:31:04.985353 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c13b9d-cbf3-4931-a6e8-c5fbed077b47 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:31:04.985353 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c13b9d-cbf3-4931-a6e8-c5fbed077b47 admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:31:04.999105 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1374/2724] 10.222.0.23 () {58 vars in 1152 bytes} [Thu May 19 12:31:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:31:05.004732 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2eb1a6-3e75-41f0-9b56-2a672009a3c3 None None] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:31:05.024341 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2eb1a6-3e75-41f0-9b56-2a672009a3c3 None None] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:31:05.029965 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fad0cd9-90f3-4ac8-95fb-fe516a376d59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:31:05.030652 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fad0cd9-90f3-4ac8-95fb-fe516a376d59 admin admin] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:31:05.030958 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fad0cd9-90f3-4ac8-95fb-fe516a376d59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:31:05.031213 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fad0cd9-90f3-4ac8-95fb-fe516a376d59 admin admin] PATH_INFO: `/v3/projects` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:31:05.032047 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fad0cd9-90f3-4ac8-95fb-fe516a376d59 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:31:05.034033 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fad0cd9-90f3-4ac8-95fb-fe516a376d59 admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:31:05.222287 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1351/2725] 10.222.0.23 () {60 vars in 1152 bytes} [Thu May 19 12:31:05 2022] POST /identity/v3/projects => generated 375 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 12:31:05.225714 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c13b9d-cbf3-4931-a6e8-c5fbed077b47 admin admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:31:05.239390 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c13b9d-cbf3-4931-a6e8-c5fbed077b47 admin admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:31:05.309014 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5183770-9e97-46ae-9054-7a64b0b48ebc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:31:05.309934 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5183770-9e97-46ae-9054-7a64b0b48ebc admin admin] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:31:05.310217 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5183770-9e97-46ae-9054-7a64b0b48ebc admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:31:05.310492 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5183770-9e97-46ae-9054-7a64b0b48ebc admin admin] PATH_INFO: `/v3/users` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:31:05.311420 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5183770-9e97-46ae-9054-7a64b0b48ebc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:31:05.313215 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5183770-9e97-46ae-9054-7a64b0b48ebc admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:31:05.461159 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1375/2726] 10.222.0.23 () {60 vars in 1143 bytes} [Thu May 19 12:31:05 2022] POST /identity/v3/users => generated 288 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 12:31:05.465890 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fad0cd9-90f3-4ac8-95fb-fe516a376d59 admin admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:31:05.477301 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fad0cd9-90f3-4ac8-95fb-fe516a376d59 admin admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:31:05.485360 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46447eea-e102-48ad-a7ef-f85886238e90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:31:05.486497 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46447eea-e102-48ad-a7ef-f85886238e90 admin admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:31:05.486999 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46447eea-e102-48ad-a7ef-f85886238e90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:31:05.487753 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46447eea-e102-48ad-a7ef-f85886238e90 admin admin] PATH_INFO: `/v3/roles` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:31:05.489201 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46447eea-e102-48ad-a7ef-f85886238e90 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:31:05.492653 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46447eea-e102-48ad-a7ef-f85886238e90 admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:31:05.514376 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1352/2727] 10.222.0.23 () {58 vars in 1121 bytes} [Thu May 19 12:31:05 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 19 12:31:05.518952 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5183770-9e97-46ae-9054-7a64b0b48ebc admin admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:31:05.530403 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5183770-9e97-46ae-9054-7a64b0b48ebc admin admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:31:05.535406 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ace6df-ca58-4e1b-9ec4-bf3a094c458d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:31:05.542848 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ace6df-ca58-4e1b-9ec4-bf3a094c458d admin admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:31:05.542848 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ace6df-ca58-4e1b-9ec4-bf3a094c458d admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:31:05.542848 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ace6df-ca58-4e1b-9ec4-bf3a094c458d admin admin] PATH_INFO: `/v3/roles` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:31:05.542848 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ace6df-ca58-4e1b-9ec4-bf3a094c458d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:31:05.542848 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ace6df-ca58-4e1b-9ec4-bf3a094c458d admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:31:05.552514 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1376/2728] 10.222.0.23 () {58 vars in 1121 bytes} [Thu May 19 12:31:05 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 12:31:05.559982 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46447eea-e102-48ad-a7ef-f85886238e90 admin admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:31:05.578126 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46447eea-e102-48ad-a7ef-f85886238e90 admin admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:31:05.589557 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f96cc3b-c5ff-4812-8f3c-a80b13747840 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:31:05.592803 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f96cc3b-c5ff-4812-8f3c-a80b13747840 admin admin] REQUEST_METHOD: `PUT` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:31:05.593321 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f96cc3b-c5ff-4812-8f3c-a80b13747840 admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:31:05.593949 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f96cc3b-c5ff-4812-8f3c-a80b13747840 admin admin] PATH_INFO: `/v3/projects/03c8ec22ac914f63a81e67c98d018815/users/d9922026c1784ed4bd40bbb5c139cf04/roles/259eb400cdb4439db72d14456ae9cd08` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:31:05.619825 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f96cc3b-c5ff-4812-8f3c-a80b13747840 admin admin] RBAC: Authorizing `identity:create_grant(project_id=03c8ec22ac914f63a81e67c98d018815, user_id=d9922026c1784ed4bd40bbb5c139cf04, role_id=259eb400cdb4439db72d14456ae9cd08)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:31:05.623270 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f96cc3b-c5ff-4812-8f3c-a80b13747840 admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:31:05.759481 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1353/2729] 10.222.0.23 () {60 vars in 1482 bytes} [Thu May 19 12:31:05 2022] PUT /identity/v3/projects/03c8ec22ac914f63a81e67c98d018815/users/d9922026c1784ed4bd40bbb5c139cf04/roles/259eb400cdb4439db72d14456ae9cd08 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:31:05.829657 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145573af-bd55-4628-a6b2-f880ecae3789 None None] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:31:05.830251 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145573af-bd55-4628-a6b2-f880ecae3789 None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:31:05.830251 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145573af-bd55-4628-a6b2-f880ecae3789 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:31:05.859764 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:31:05.865253 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.auth.core [None req-145573af-bd55-4628-a6b2-f880ecae3789 None None] MFA Rules not processed for user `d9922026c1784ed4bd40bbb5c139cf04`. Rule list: `[]` (Enabled: `True`). {{(pid=80540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:31:05.866650 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-145573af-bd55-4628-a6b2-f880ecae3789 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:31:05.992256 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1377/2730] 10.222.0.23 () {58 vars in 957 bytes} [Thu May 19 12:31:05 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:31:06.009851 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f96cc3b-c5ff-4812-8f3c-a80b13747840 admin admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:31:06.012065 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-5f96cc3b-c5ff-4812-8f3c-a80b13747840 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:31:06.098357 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f96cc3b-c5ff-4812-8f3c-a80b13747840 admin admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:31:06.105368 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9ba97d-eb00-4309-8177-20975ab8be89 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8800254c0e784e42a791fb9abc274d79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd70d25bf69314820b8aaecb65576978a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:31:06.106530 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9ba97d-eb00-4309-8177-20975ab8be89 service cinder] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:31:06.107043 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9ba97d-eb00-4309-8177-20975ab8be89 service cinder] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:31:06.107554 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9ba97d-eb00-4309-8177-20975ab8be89 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:31:06.124922 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9ba97d-eb00-4309-8177-20975ab8be89 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:31:06.127392 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9ba97d-eb00-4309-8177-20975ab8be89 service cinder] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:31:06.146924 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1354/2731] 10.222.0.23 () {60 vars in 1365 bytes} [Thu May 19 12:31:06 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:31:09.148805 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95420caa-dffe-4bf5-bb28-219cbc7c9975 None None] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:31:09.148805 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95420caa-dffe-4bf5-bb28-219cbc7c9975 None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:31:09.148805 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95420caa-dffe-4bf5-bb28-219cbc7c9975 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:31:09.148805 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.auth.core [None req-95420caa-dffe-4bf5-bb28-219cbc7c9975 None None] MFA Rules not processed for user `d9922026c1784ed4bd40bbb5c139cf04`. Rule list: `[]` (Enabled: `True`). {{(pid=80540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:31:09.148805 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-95420caa-dffe-4bf5-bb28-219cbc7c9975 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:31:09.148805 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1378/2732] 10.222.0.23 () {58 vars in 957 bytes} [Thu May 19 12:31:09 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:31:14.798097 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9ba97d-eb00-4309-8177-20975ab8be89 service cinder] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:31:14.800101 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-cd9ba97d-eb00-4309-8177-20975ab8be89 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:31:14.898176 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9ba97d-eb00-4309-8177-20975ab8be89 service cinder] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:31:14.904395 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0674f2f-7136-475c-9e93-0006d09b20e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:31:14.905290 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0674f2f-7136-475c-9e93-0006d09b20e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:31:14.905584 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0674f2f-7136-475c-9e93-0006d09b20e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:31:14.905857 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0674f2f-7136-475c-9e93-0006d09b20e6 admin admin] PATH_INFO: `/v3/users/0f9a99fbce9b4de08cd1662f5ddeaedc` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:31:14.907136 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0674f2f-7136-475c-9e93-0006d09b20e6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0f9a99fbce9b4de08cd1662f5ddeaedc)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:31:14.909156 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0674f2f-7136-475c-9e93-0006d09b20e6 admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:31:15.203181 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=93,keystone:INSERT=2,keystone:DELETE=5 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:31:15.211083 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-f0674f2f-7136-475c-9e93-0006d09b20e6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0f9a99fbce9b4de08cd1662f5ddeaedc'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:31:15.227701 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-f0674f2f-7136-475c-9e93-0006d09b20e6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0f9a99fbce9b4de08cd1662f5ddeaedc'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:31:15.229100 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-f0674f2f-7136-475c-9e93-0006d09b20e6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0f9a99fbce9b4de08cd1662f5ddeaedc'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:31:15.307387 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-f0674f2f-7136-475c-9e93-0006d09b20e6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0f9a99fbce9b4de08cd1662f5ddeaedc'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:31:15.317622 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-f0674f2f-7136-475c-9e93-0006d09b20e6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0f9a99fbce9b4de08cd1662f5ddeaedc'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:31:15.327424 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1355/2733] 10.222.0.23 () {58 vars in 1223 bytes} [Thu May 19 12:31:14 2022] DELETE /identity/v3/users/0f9a99fbce9b4de08cd1662f5ddeaedc => generated 0 bytes in 530 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:31:15.334063 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a1bfa2-e4de-48df-b3f4-7f139108b9ad None None] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:31:15.334624 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a1bfa2-e4de-48df-b3f4-7f139108b9ad None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:31:15.334985 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a1bfa2-e4de-48df-b3f4-7f139108b9ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:31:15.364603 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.auth.core [None req-19a1bfa2-e4de-48df-b3f4-7f139108b9ad None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:31:15.366159 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-19a1bfa2-e4de-48df-b3f4-7f139108b9ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:31:15.475362 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1379/2734] 10.222.0.23 () {58 vars in 957 bytes} [Thu May 19 12:31:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:31:15.488653 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0674f2f-7136-475c-9e93-0006d09b20e6 admin admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:31:15.491172 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-f0674f2f-7136-475c-9e93-0006d09b20e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:31:15.564827 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0674f2f-7136-475c-9e93-0006d09b20e6 admin admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:31:15.569481 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97af009-edb6-40bb-9bcf-5bdf5e7a4ffa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a00b3fa7497456895e5e0bf105c4fc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd70d25bf69314820b8aaecb65576978a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:31:15.570154 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97af009-edb6-40bb-9bcf-5bdf5e7a4ffa service neutron] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:31:15.570432 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97af009-edb6-40bb-9bcf-5bdf5e7a4ffa service neutron] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:31:15.570746 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97af009-edb6-40bb-9bcf-5bdf5e7a4ffa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:31:15.589021 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97af009-edb6-40bb-9bcf-5bdf5e7a4ffa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:31:15.590362 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97af009-edb6-40bb-9bcf-5bdf5e7a4ffa service neutron] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:31:15.602368 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1356/2735] 10.222.0.23 () {60 vars in 1365 bytes} [Thu May 19 12:31:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:31:16.247518 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a1bfa2-e4de-48df-b3f4-7f139108b9ad None None] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:31:16.250009 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-19a1bfa2-e4de-48df-b3f4-7f139108b9ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:31:16.266485 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a1bfa2-e4de-48df-b3f4-7f139108b9ad None None] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:31:16.272692 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5490a201-e731-4e52-b33a-5734eeb97232 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:31:16.273610 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5490a201-e731-4e52-b33a-5734eeb97232 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:31:16.273896 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5490a201-e731-4e52-b33a-5734eeb97232 admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:31:16.274178 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5490a201-e731-4e52-b33a-5734eeb97232 admin admin] PATH_INFO: `/v3/projects/b96f24f462014ae6a3dbf90008e7579f` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:31:16.277143 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5490a201-e731-4e52-b33a-5734eeb97232 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b96f24f462014ae6a3dbf90008e7579f)` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:31:16.279482 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5490a201-e731-4e52-b33a-5734eeb97232 admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:31:16.419985 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-5490a201-e731-4e52-b33a-5734eeb97232 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b96f24f462014ae6a3dbf90008e7579f', 'request_id': 'req-5490a201-e731-4e52-b33a-5734eeb97232'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:31:16.427342 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-5490a201-e731-4e52-b33a-5734eeb97232 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b96f24f462014ae6a3dbf90008e7579f', 'request_id': 'req-5490a201-e731-4e52-b33a-5734eeb97232'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:31:16.438465 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-5490a201-e731-4e52-b33a-5734eeb97232 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b96f24f462014ae6a3dbf90008e7579f', 'request_id': 'req-5490a201-e731-4e52-b33a-5734eeb97232'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:31:16.569154 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-5490a201-e731-4e52-b33a-5734eeb97232 admin admin] The token cache is being invalidate because project b96f24f462014ae6a3dbf90008e7579f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80540) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 12:31:16.569476 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-5490a201-e731-4e52-b33a-5734eeb97232 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:31:16.570173 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-5490a201-e731-4e52-b33a-5734eeb97232 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:31:16.572784 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1380/2736] 10.222.0.23 () {58 vars in 1232 bytes} [Thu May 19 12:31:16 2022] DELETE /identity/v3/projects/b96f24f462014ae6a3dbf90008e7579f => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:31:16.577231 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97af009-edb6-40bb-9bcf-5bdf5e7a4ffa service neutron] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:31:16.588702 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97af009-edb6-40bb-9bcf-5bdf5e7a4ffa service neutron] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:31:16.652798 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c1e852-53d6-4019-9b5b-d3ea5fa3b787 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:31:16.654056 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c1e852-53d6-4019-9b5b-d3ea5fa3b787 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:31:16.654464 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c1e852-53d6-4019-9b5b-d3ea5fa3b787 admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:31:16.654877 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c1e852-53d6-4019-9b5b-d3ea5fa3b787 admin admin] PATH_INFO: `/v3/users/af93cbdeee3042078fa7782f6a6d1e63` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:31:16.656546 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c1e852-53d6-4019-9b5b-d3ea5fa3b787 admin admin] RBAC: Authorizing `identity:delete_user(user_id=af93cbdeee3042078fa7782f6a6d1e63)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:31:16.661081 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c1e852-53d6-4019-9b5b-d3ea5fa3b787 admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:31:16.912605 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-15c1e852-53d6-4019-9b5b-d3ea5fa3b787 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'af93cbdeee3042078fa7782f6a6d1e63'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:31:16.925217 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:31:16.935442 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-15c1e852-53d6-4019-9b5b-d3ea5fa3b787 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'af93cbdeee3042078fa7782f6a6d1e63'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:31:16.936577 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-15c1e852-53d6-4019-9b5b-d3ea5fa3b787 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'af93cbdeee3042078fa7782f6a6d1e63'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:31:17.248225 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-15c1e852-53d6-4019-9b5b-d3ea5fa3b787 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'af93cbdeee3042078fa7782f6a6d1e63'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:31:17.257544 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-15c1e852-53d6-4019-9b5b-d3ea5fa3b787 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'af93cbdeee3042078fa7782f6a6d1e63'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:31:17.268273 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1357/2737] 10.222.0.23 () {58 vars in 1223 bytes} [Thu May 19 12:31:16 2022] DELETE /identity/v3/users/af93cbdeee3042078fa7782f6a6d1e63 => generated 0 bytes in 689 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:31:17.516252 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5490a201-e731-4e52-b33a-5734eeb97232 admin admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:31:17.518216 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-5490a201-e731-4e52-b33a-5734eeb97232 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:31:17.532245 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5490a201-e731-4e52-b33a-5734eeb97232 admin admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:31:17.572175 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:31:17.658940 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9a5746-e7d2-4080-942a-9fcf2dcfe8ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:31:17.660547 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9a5746-e7d2-4080-942a-9fcf2dcfe8ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:31:17.661052 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9a5746-e7d2-4080-942a-9fcf2dcfe8ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:31:17.661540 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9a5746-e7d2-4080-942a-9fcf2dcfe8ba admin admin] PATH_INFO: `/v3/users/d3bd602bec704b01b66ab9bb6afd5ff5` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:31:17.664299 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9a5746-e7d2-4080-942a-9fcf2dcfe8ba admin admin] RBAC: Authorizing `identity:delete_user(user_id=d3bd602bec704b01b66ab9bb6afd5ff5)` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:31:17.668094 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9a5746-e7d2-4080-942a-9fcf2dcfe8ba admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:31:17.876341 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c1e852-53d6-4019-9b5b-d3ea5fa3b787 admin admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:31:17.878160 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-15c1e852-53d6-4019-9b5b-d3ea5fa3b787 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:31:17.893658 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c1e852-53d6-4019-9b5b-d3ea5fa3b787 admin admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:31:17.903719 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2710d74a-0c54-4fd3-9805-719aa27063bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:31:17.905913 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2710d74a-0c54-4fd3-9805-719aa27063bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:31:17.906756 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2710d74a-0c54-4fd3-9805-719aa27063bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:31:17.907465 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2710d74a-0c54-4fd3-9805-719aa27063bb admin admin] PATH_INFO: `/v3/projects/943f983348f54f508101cb946a8fd2f1` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:31:17.909502 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2710d74a-0c54-4fd3-9805-719aa27063bb admin admin] RBAC: Authorizing `identity:delete_project(project_id=943f983348f54f508101cb946a8fd2f1)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:31:17.912428 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2710d74a-0c54-4fd3-9805-719aa27063bb admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:31:17.990634 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-6e9a5746-e7d2-4080-942a-9fcf2dcfe8ba admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd3bd602bec704b01b66ab9bb6afd5ff5'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:31:18.016750 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-6e9a5746-e7d2-4080-942a-9fcf2dcfe8ba admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd3bd602bec704b01b66ab9bb6afd5ff5'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:31:18.018099 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-6e9a5746-e7d2-4080-942a-9fcf2dcfe8ba admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd3bd602bec704b01b66ab9bb6afd5ff5'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:31:18.118547 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-6e9a5746-e7d2-4080-942a-9fcf2dcfe8ba admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd3bd602bec704b01b66ab9bb6afd5ff5'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:31:18.131843 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-2710d74a-0c54-4fd3-9805-719aa27063bb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '943f983348f54f508101cb946a8fd2f1', 'request_id': 'req-2710d74a-0c54-4fd3-9805-719aa27063bb'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:31:18.138526 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-6e9a5746-e7d2-4080-942a-9fcf2dcfe8ba admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd3bd602bec704b01b66ab9bb6afd5ff5'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:31:18.141870 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-2710d74a-0c54-4fd3-9805-719aa27063bb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '943f983348f54f508101cb946a8fd2f1', 'request_id': 'req-2710d74a-0c54-4fd3-9805-719aa27063bb'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:31:18.157180 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1381/2738] 10.222.0.23 () {58 vars in 1223 bytes} [Thu May 19 12:31:17 2022] DELETE /identity/v3/users/d3bd602bec704b01b66ab9bb6afd5ff5 => generated 0 bytes in 641 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 12:31:18.162856 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fab51a0-b4b9-48a1-a084-db0f02554928 None None] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:31:18.163524 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fab51a0-b4b9-48a1-a084-db0f02554928 None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:31:18.164061 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fab51a0-b4b9-48a1-a084-db0f02554928 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:31:18.186338 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.auth.core [None req-1fab51a0-b4b9-48a1-a084-db0f02554928 None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:31:18.187233 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-1fab51a0-b4b9-48a1-a084-db0f02554928 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:31:18.211957 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-2710d74a-0c54-4fd3-9805-719aa27063bb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '943f983348f54f508101cb946a8fd2f1', 'request_id': 'req-2710d74a-0c54-4fd3-9805-719aa27063bb'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:31:18.233047 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-2710d74a-0c54-4fd3-9805-719aa27063bb admin admin] The token cache is being invalidate because project 943f983348f54f508101cb946a8fd2f1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80539) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 12:31:18.234181 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-2710d74a-0c54-4fd3-9805-719aa27063bb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:31:18.236804 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-2710d74a-0c54-4fd3-9805-719aa27063bb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:31:18.248439 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1358/2739] 10.222.0.23 () {58 vars in 1232 bytes} [Thu May 19 12:31:17 2022] DELETE /identity/v3/projects/943f983348f54f508101cb946a8fd2f1 => generated 0 bytes in 366 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 12:31:18.252248 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70c90c2-6d82-431d-9370-87152161130b None None] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:31:18.252248 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70c90c2-6d82-431d-9370-87152161130b None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:31:18.258577 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70c90c2-6d82-431d-9370-87152161130b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:31:18.293433 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.auth.core [None req-b70c90c2-6d82-431d-9370-87152161130b None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:31:18.298653 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-b70c90c2-6d82-431d-9370-87152161130b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:31:18.315362 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1382/2740] 10.222.0.23 () {58 vars in 957 bytes} [Thu May 19 12:31:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 12:31:18.331202 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fab51a0-b4b9-48a1-a084-db0f02554928 None None] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:31:18.333897 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-1fab51a0-b4b9-48a1-a084-db0f02554928 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:31:18.409550 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1359/2741] 10.222.0.23 () {58 vars in 957 bytes} [Thu May 19 12:31:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:31:18.414869 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58249ed-a0d3-43c4-a8a6-cc6eaac89c34 None None] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:31:18.415177 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58249ed-a0d3-43c4-a8a6-cc6eaac89c34 None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:31:18.415491 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58249ed-a0d3-43c4-a8a6-cc6eaac89c34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:31:18.437714 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.auth.core [None req-c58249ed-a0d3-43c4-a8a6-cc6eaac89c34 None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:31:18.438971 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-c58249ed-a0d3-43c4-a8a6-cc6eaac89c34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:31:18.442986 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:31:18.453800 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1360/2742] 10.222.0.23 () {58 vars in 957 bytes} [Thu May 19 12:31:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 12:31:18.458910 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58249ed-a0d3-43c4-a8a6-cc6eaac89c34 None None] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:31:18.465911 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fab51a0-b4b9-48a1-a084-db0f02554928 None None] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:31:18.472999 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676e5031-2480-4a50-9e7b-df3213fb8091 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a00b3fa7497456895e5e0bf105c4fc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd70d25bf69314820b8aaecb65576978a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:31:18.474161 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676e5031-2480-4a50-9e7b-df3213fb8091 service neutron] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:31:18.474706 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676e5031-2480-4a50-9e7b-df3213fb8091 service neutron] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:31:18.475257 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676e5031-2480-4a50-9e7b-df3213fb8091 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:31:18.475498 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58249ed-a0d3-43c4-a8a6-cc6eaac89c34 None None] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:31:18.483231 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1bad54-db48-4b72-be48-48485e0668dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:31:18.484359 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1bad54-db48-4b72-be48-48485e0668dc admin admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:31:18.484816 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1bad54-db48-4b72-be48-48485e0668dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:31:18.485271 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1bad54-db48-4b72-be48-48485e0668dc admin admin] PATH_INFO: `/v3/domains` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:31:18.486469 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1bad54-db48-4b72-be48-48485e0668dc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:31:18.487198 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1bad54-db48-4b72-be48-48485e0668dc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:31:18.490223 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1bad54-db48-4b72-be48-48485e0668dc admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:31:18.492283 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-676e5031-2480-4a50-9e7b-df3213fb8091 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:31:18.495255 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-676e5031-2480-4a50-9e7b-df3213fb8091 service neutron] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:31:18.507193 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1361/2743] 10.222.0.23 () {58 vars in 1152 bytes} [Thu May 19 12:31:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 12:31:18.507714 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1383/2744] 10.222.0.23 () {60 vars in 1365 bytes} [Thu May 19 12:31:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 12:31:18.511812 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1bad54-db48-4b72-be48-48485e0668dc admin admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:31:18.523593 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1bad54-db48-4b72-be48-48485e0668dc admin admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:31:18.532262 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a2fd79-9fbb-4f55-a48b-90a7e8a89647 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:31:18.535410 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a2fd79-9fbb-4f55-a48b-90a7e8a89647 admin admin] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:31:18.535963 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a2fd79-9fbb-4f55-a48b-90a7e8a89647 admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:31:18.536484 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a2fd79-9fbb-4f55-a48b-90a7e8a89647 admin admin] PATH_INFO: `/v3/projects` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:31:18.537834 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a2fd79-9fbb-4f55-a48b-90a7e8a89647 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:31:18.541709 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a2fd79-9fbb-4f55-a48b-90a7e8a89647 admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:31:18.668462 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1362/2745] 10.222.0.23 () {60 vars in 1152 bytes} [Thu May 19 12:31:18 2022] POST /identity/v3/projects => generated 385 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 12:31:18.673637 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676e5031-2480-4a50-9e7b-df3213fb8091 service neutron] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:31:18.689256 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676e5031-2480-4a50-9e7b-df3213fb8091 service neutron] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:31:18.762652 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77412619-6716-49b5-9962-121e13631a2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:31:18.763349 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77412619-6716-49b5-9962-121e13631a2b admin admin] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:31:18.763640 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77412619-6716-49b5-9962-121e13631a2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:31:18.764037 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77412619-6716-49b5-9962-121e13631a2b admin admin] PATH_INFO: `/v3/users` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:31:18.764740 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77412619-6716-49b5-9962-121e13631a2b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:31:18.766744 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77412619-6716-49b5-9962-121e13631a2b admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:31:18.868645 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1384/2746] 10.222.0.23 () {60 vars in 1143 bytes} [Thu May 19 12:31:18 2022] POST /identity/v3/users => generated 293 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 12:31:18.874086 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a2fd79-9fbb-4f55-a48b-90a7e8a89647 admin admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:31:18.885410 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a2fd79-9fbb-4f55-a48b-90a7e8a89647 admin admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:31:18.891612 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42582022-97de-4af9-b872-12c42dc607ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:31:18.892680 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42582022-97de-4af9-b872-12c42dc607ca admin admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:31:18.893071 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42582022-97de-4af9-b872-12c42dc607ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:31:18.893602 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42582022-97de-4af9-b872-12c42dc607ca admin admin] PATH_INFO: `/v3/roles` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:31:18.894628 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42582022-97de-4af9-b872-12c42dc607ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:31:18.896455 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42582022-97de-4af9-b872-12c42dc607ca admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:31:18.910624 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1363/2747] 10.222.0.23 () {58 vars in 1121 bytes} [Thu May 19 12:31:18 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 12:31:18.916084 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77412619-6716-49b5-9962-121e13631a2b admin admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:31:18.932191 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77412619-6716-49b5-9962-121e13631a2b admin admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:31:18.940242 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5be42a-a2ea-4703-b848-561bb04feacf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:31:18.941205 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5be42a-a2ea-4703-b848-561bb04feacf admin admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:31:18.941551 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5be42a-a2ea-4703-b848-561bb04feacf admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:31:18.941838 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5be42a-a2ea-4703-b848-561bb04feacf admin admin] PATH_INFO: `/v3/roles` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:31:18.942956 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5be42a-a2ea-4703-b848-561bb04feacf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:31:18.945285 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5be42a-a2ea-4703-b848-561bb04feacf admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:31:18.975835 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1385/2748] 10.222.0.23 () {58 vars in 1121 bytes} [Thu May 19 12:31:18 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 12:31:18.981323 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42582022-97de-4af9-b872-12c42dc607ca admin admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:31:18.995679 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42582022-97de-4af9-b872-12c42dc607ca admin admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:31:19.001328 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec783ed0-c1e6-4603-a6d4-bec713f50fdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:31:19.002355 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec783ed0-c1e6-4603-a6d4-bec713f50fdf admin admin] REQUEST_METHOD: `PUT` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:31:19.002647 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec783ed0-c1e6-4603-a6d4-bec713f50fdf admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:31:19.002933 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec783ed0-c1e6-4603-a6d4-bec713f50fdf admin admin] PATH_INFO: `/v3/projects/f386834878664344a889044da5df964a/users/607c1b5e793647b78fb4f7a9222ab5c8/roles/259eb400cdb4439db72d14456ae9cd08` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:31:19.006533 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:DELETE=9,keystone:SELECT=88,keystone:INSERT=3 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:31:19.021455 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec783ed0-c1e6-4603-a6d4-bec713f50fdf admin admin] RBAC: Authorizing `identity:create_grant(project_id=f386834878664344a889044da5df964a, user_id=607c1b5e793647b78fb4f7a9222ab5c8, role_id=259eb400cdb4439db72d14456ae9cd08)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:31:19.024527 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec783ed0-c1e6-4603-a6d4-bec713f50fdf admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:31:19.143344 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5be42a-a2ea-4703-b848-561bb04feacf admin admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:31:19.148449 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-8b5be42a-a2ea-4703-b848-561bb04feacf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:31:19.174992 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5be42a-a2ea-4703-b848-561bb04feacf admin admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:31:19.185117 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1364/2749] 10.222.0.23 () {60 vars in 1482 bytes} [Thu May 19 12:31:18 2022] PUT /identity/v3/projects/f386834878664344a889044da5df964a/users/607c1b5e793647b78fb4f7a9222ab5c8/roles/259eb400cdb4439db72d14456ae9cd08 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:31:19.204937 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d2950b-191d-4186-84b2-c86aa90b32a8 None None] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:31:19.205895 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d2950b-191d-4186-84b2-c86aa90b32a8 None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:31:19.206734 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d2950b-191d-4186-84b2-c86aa90b32a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:31:19.254643 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.auth.core [None req-66d2950b-191d-4186-84b2-c86aa90b32a8 None None] MFA Rules not processed for user `607c1b5e793647b78fb4f7a9222ab5c8`. Rule list: `[]` (Enabled: `True`). {{(pid=80539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:31:19.255914 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-66d2950b-191d-4186-84b2-c86aa90b32a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:31:19.296727 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeefe792-6806-4edc-a6e0-0feaf55ae3ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:31:19.297852 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeefe792-6806-4edc-a6e0-0feaf55ae3ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:31:19.298316 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeefe792-6806-4edc-a6e0-0feaf55ae3ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:31:19.298765 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeefe792-6806-4edc-a6e0-0feaf55ae3ab admin admin] PATH_INFO: `/v3/projects/1878428dc5ae445eb085c999050cea53` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:31:19.300570 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeefe792-6806-4edc-a6e0-0feaf55ae3ab admin admin] RBAC: Authorizing `identity:delete_project(project_id=1878428dc5ae445eb085c999050cea53)` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:31:19.303172 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeefe792-6806-4edc-a6e0-0feaf55ae3ab admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:31:19.346573 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1365/2750] 10.222.0.23 () {58 vars in 957 bytes} [Thu May 19 12:31:19 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 12:31:19.351914 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d2950b-191d-4186-84b2-c86aa90b32a8 None None] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:31:19.365384 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d2950b-191d-4186-84b2-c86aa90b32a8 None None] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:31:19.370696 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30ac9d3-2fb4-4e11-9b12-944f6d94dd20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:31:19.371372 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30ac9d3-2fb4-4e11-9b12-944f6d94dd20 admin admin] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:31:19.371662 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30ac9d3-2fb4-4e11-9b12-944f6d94dd20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:31:19.371973 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30ac9d3-2fb4-4e11-9b12-944f6d94dd20 admin admin] PATH_INFO: `/v3/projects` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:31:19.372732 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30ac9d3-2fb4-4e11-9b12-944f6d94dd20 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:31:19.374728 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30ac9d3-2fb4-4e11-9b12-944f6d94dd20 admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:31:19.474735 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1366/2751] 10.222.0.23 () {60 vars in 1152 bytes} [Thu May 19 12:31:19 2022] POST /identity/v3/projects => generated 383 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 12:31:19.484279 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30ac9d3-2fb4-4e11-9b12-944f6d94dd20 admin admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:31:19.509847 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-eeefe792-6806-4edc-a6e0-0feaf55ae3ab admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1878428dc5ae445eb085c999050cea53', 'request_id': 'req-eeefe792-6806-4edc-a6e0-0feaf55ae3ab'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:31:19.526796 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-eeefe792-6806-4edc-a6e0-0feaf55ae3ab admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1878428dc5ae445eb085c999050cea53', 'request_id': 'req-eeefe792-6806-4edc-a6e0-0feaf55ae3ab'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:31:19.534461 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-eeefe792-6806-4edc-a6e0-0feaf55ae3ab admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1878428dc5ae445eb085c999050cea53', 'request_id': 'req-eeefe792-6806-4edc-a6e0-0feaf55ae3ab'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:31:19.632048 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-eeefe792-6806-4edc-a6e0-0feaf55ae3ab admin admin] The token cache is being invalidate because project 1878428dc5ae445eb085c999050cea53 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80540) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 12:31:19.633125 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-eeefe792-6806-4edc-a6e0-0feaf55ae3ab admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:31:19.634756 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-eeefe792-6806-4edc-a6e0-0feaf55ae3ab admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:31:19.637586 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=91,keystone:INSERT=3,keystone:DELETE=6 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:31:19.638942 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30ac9d3-2fb4-4e11-9b12-944f6d94dd20 admin admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:31:19.645090 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1386/2752] 10.222.0.23 () {58 vars in 1232 bytes} [Thu May 19 12:31:19 2022] DELETE /identity/v3/projects/1878428dc5ae445eb085c999050cea53 => generated 0 bytes in 502 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 12:31:19.649570 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeefe792-6806-4edc-a6e0-0feaf55ae3ab admin admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:31:19.652848 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-eeefe792-6806-4edc-a6e0-0feaf55ae3ab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:31:19.735730 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61177dc-0160-49c1-a74d-004fcb815a82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:31:19.736582 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61177dc-0160-49c1-a74d-004fcb815a82 admin admin] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:31:19.736879 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61177dc-0160-49c1-a74d-004fcb815a82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:31:19.737160 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61177dc-0160-49c1-a74d-004fcb815a82 admin admin] PATH_INFO: `/v3/users` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:31:19.738134 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d61177dc-0160-49c1-a74d-004fcb815a82 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:31:19.740210 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d61177dc-0160-49c1-a74d-004fcb815a82 admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:31:19.753005 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeefe792-6806-4edc-a6e0-0feaf55ae3ab admin admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:31:19.758682 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da721c53-f0ee-4381-839d-48dc22775808 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:31:19.761274 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da721c53-f0ee-4381-839d-48dc22775808 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:31:19.761565 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da721c53-f0ee-4381-839d-48dc22775808 admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:31:19.761847 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da721c53-f0ee-4381-839d-48dc22775808 admin admin] PATH_INFO: `/v3/users/7d54049a8cf14872a6ec27687aaa5964` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:31:19.763555 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da721c53-f0ee-4381-839d-48dc22775808 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7d54049a8cf14872a6ec27687aaa5964)` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:31:19.765997 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da721c53-f0ee-4381-839d-48dc22775808 admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:31:19.851479 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1367/2753] 10.222.0.23 () {60 vars in 1143 bytes} [Thu May 19 12:31:19 2022] POST /identity/v3/users => generated 298 bytes in 366 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 12:31:19.853167 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61177dc-0160-49c1-a74d-004fcb815a82 admin admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:31:19.855222 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-d61177dc-0160-49c1-a74d-004fcb815a82 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:31:19.869665 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61177dc-0160-49c1-a74d-004fcb815a82 admin admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:31:19.883724 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb962c8-17a2-44dd-b380-ae580c468dd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:31:19.885209 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb962c8-17a2-44dd-b380-ae580c468dd9 admin admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:31:19.885800 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb962c8-17a2-44dd-b380-ae580c468dd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:31:19.886910 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb962c8-17a2-44dd-b380-ae580c468dd9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:31:19.887892 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb962c8-17a2-44dd-b380-ae580c468dd9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:31:19.889729 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb962c8-17a2-44dd-b380-ae580c468dd9 admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:31:19.908123 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1368/2754] 10.222.0.23 () {58 vars in 1121 bytes} [Thu May 19 12:31:19 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 12:31:19.912804 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb962c8-17a2-44dd-b380-ae580c468dd9 admin admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:31:19.933184 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb962c8-17a2-44dd-b380-ae580c468dd9 admin admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:31:19.941897 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b99a89e-8098-4653-a756-aaf3cdf9ac47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:31:19.943163 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b99a89e-8098-4653-a756-aaf3cdf9ac47 admin admin] REQUEST_METHOD: `PUT` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:31:19.943697 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b99a89e-8098-4653-a756-aaf3cdf9ac47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:31:19.944096 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b99a89e-8098-4653-a756-aaf3cdf9ac47 admin admin] PATH_INFO: `/v3/projects/36bd679d2cc84ead85750d030d91df00/users/f2c58ab7ebd64355afd6bd07ba188531/roles/9db5671c65d3414ea2682afa2ccf1403` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:31:19.968885 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b99a89e-8098-4653-a756-aaf3cdf9ac47 admin admin] RBAC: Authorizing `identity:create_grant(project_id=36bd679d2cc84ead85750d030d91df00, user_id=f2c58ab7ebd64355afd6bd07ba188531, role_id=9db5671c65d3414ea2682afa2ccf1403)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:31:19.976166 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b99a89e-8098-4653-a756-aaf3cdf9ac47 admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:31:19.994328 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-da721c53-f0ee-4381-839d-48dc22775808 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7d54049a8cf14872a6ec27687aaa5964'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:31:20.018592 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-da721c53-f0ee-4381-839d-48dc22775808 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7d54049a8cf14872a6ec27687aaa5964'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:31:20.020251 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-da721c53-f0ee-4381-839d-48dc22775808 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7d54049a8cf14872a6ec27687aaa5964'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:31:20.100573 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1369/2755] 10.222.0.23 () {60 vars in 1482 bytes} [Thu May 19 12:31:19 2022] PUT /identity/v3/projects/36bd679d2cc84ead85750d030d91df00/users/f2c58ab7ebd64355afd6bd07ba188531/roles/9db5671c65d3414ea2682afa2ccf1403 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:31:20.106297 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b99a89e-8098-4653-a756-aaf3cdf9ac47 admin admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:31:20.152436 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-da721c53-f0ee-4381-839d-48dc22775808 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7d54049a8cf14872a6ec27687aaa5964'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:31:20.166841 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-da721c53-f0ee-4381-839d-48dc22775808 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7d54049a8cf14872a6ec27687aaa5964'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:31:20.175164 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1387/2756] 10.222.0.23 () {58 vars in 1223 bytes} [Thu May 19 12:31:19 2022] DELETE /identity/v3/users/7d54049a8cf14872a6ec27687aaa5964 => generated 0 bytes in 526 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:31:20.220849 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b99a89e-8098-4653-a756-aaf3cdf9ac47 admin admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:31:20.237094 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:31:20.332787 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bac84b7-6cab-429f-81d3-fc48f6502eb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:31:20.336798 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bac84b7-6cab-429f-81d3-fc48f6502eb0 admin admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:31:20.336798 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bac84b7-6cab-429f-81d3-fc48f6502eb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:31:20.336798 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bac84b7-6cab-429f-81d3-fc48f6502eb0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:31:20.336798 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bac84b7-6cab-429f-81d3-fc48f6502eb0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:31:20.336798 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bac84b7-6cab-429f-81d3-fc48f6502eb0 admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:31:20.350999 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1370/2757] 10.222.0.23 () {58 vars in 1121 bytes} [Thu May 19 12:31:20 2022] GET /identity/v3/roles => generated 1420 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 19 12:31:20.352752 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da721c53-f0ee-4381-839d-48dc22775808 admin admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:31:20.355693 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-da721c53-f0ee-4381-839d-48dc22775808 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:31:20.376321 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da721c53-f0ee-4381-839d-48dc22775808 admin admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:31:20.381893 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573d1f4f-b9de-45d0-a07a-4041cc21cd90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:31:20.382820 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573d1f4f-b9de-45d0-a07a-4041cc21cd90 admin admin] REQUEST_METHOD: `PUT` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:31:20.383100 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573d1f4f-b9de-45d0-a07a-4041cc21cd90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:31:20.383439 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573d1f4f-b9de-45d0-a07a-4041cc21cd90 admin admin] PATH_INFO: `/v3/projects/36bd679d2cc84ead85750d030d91df00/users/f2c58ab7ebd64355afd6bd07ba188531/roles/9db5671c65d3414ea2682afa2ccf1403` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:31:20.386457 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-573d1f4f-b9de-45d0-a07a-4041cc21cd90 admin admin] RBAC: Authorizing `identity:create_grant(project_id=36bd679d2cc84ead85750d030d91df00, user_id=f2c58ab7ebd64355afd6bd07ba188531, role_id=9db5671c65d3414ea2682afa2ccf1403)` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:31:20.389347 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-573d1f4f-b9de-45d0-a07a-4041cc21cd90 admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:31:20.401979 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1388/2758] 10.222.0.23 () {60 vars in 1482 bytes} [Thu May 19 12:31:20 2022] PUT /identity/v3/projects/36bd679d2cc84ead85750d030d91df00/users/f2c58ab7ebd64355afd6bd07ba188531/roles/9db5671c65d3414ea2682afa2ccf1403 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:31:20.413960 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383f88c2-8219-4b45-9eef-9960cc1cc822 None None] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:31:20.414299 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383f88c2-8219-4b45-9eef-9960cc1cc822 None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:31:20.414623 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383f88c2-8219-4b45-9eef-9960cc1cc822 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:31:20.473229 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.auth.core [None req-383f88c2-8219-4b45-9eef-9960cc1cc822 None None] MFA Rules not processed for user `f2c58ab7ebd64355afd6bd07ba188531`. Rule list: `[]` (Enabled: `True`). {{(pid=80539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:31:20.479038 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-383f88c2-8219-4b45-9eef-9960cc1cc822 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:31:20.596369 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1371/2759] 10.222.0.23 () {58 vars in 957 bytes} [Thu May 19 12:31:20 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:31:20.618066 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573d1f4f-b9de-45d0-a07a-4041cc21cd90 admin admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:31:20.623369 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-573d1f4f-b9de-45d0-a07a-4041cc21cd90 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:31:20.681859 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:31:20.757481 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573d1f4f-b9de-45d0-a07a-4041cc21cd90 admin admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:31:20.764779 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70ab835-8270-4c80-ae8f-3fd23f514594 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8800254c0e784e42a791fb9abc274d79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd70d25bf69314820b8aaecb65576978a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:31:20.768065 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70ab835-8270-4c80-ae8f-3fd23f514594 service cinder] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:31:20.768065 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70ab835-8270-4c80-ae8f-3fd23f514594 service cinder] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:31:20.768065 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70ab835-8270-4c80-ae8f-3fd23f514594 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:31:20.770621 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-b70ab835-8270-4c80-ae8f-3fd23f514594 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:31:20.791856 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:INSERT=3,keystone:DELETE=11,keystone:SELECT=86 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:31:20.833943 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383f88c2-8219-4b45-9eef-9960cc1cc822 None None] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:31:20.835400 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-383f88c2-8219-4b45-9eef-9960cc1cc822 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:31:20.903833 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70ab835-8270-4c80-ae8f-3fd23f514594 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:31:20.905785 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70ab835-8270-4c80-ae8f-3fd23f514594 service cinder] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:31:20.922075 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1389/2760] 10.222.0.23 () {60 vars in 1365 bytes} [Thu May 19 12:31:20 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:31:20.960374 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383f88c2-8219-4b45-9eef-9960cc1cc822 None None] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:31:20.974724 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9523bb24-91df-4ab9-b86f-60663f1f5331 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:31:20.976991 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9523bb24-91df-4ab9-b86f-60663f1f5331 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:31:20.977785 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9523bb24-91df-4ab9-b86f-60663f1f5331 admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:31:20.981072 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9523bb24-91df-4ab9-b86f-60663f1f5331 admin admin] PATH_INFO: `/v3/projects/178730f5589f472da96cc280c8b80c9c` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:31:20.984075 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9523bb24-91df-4ab9-b86f-60663f1f5331 admin admin] RBAC: Authorizing `identity:delete_project(project_id=178730f5589f472da96cc280c8b80c9c)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:31:20.987587 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9523bb24-91df-4ab9-b86f-60663f1f5331 admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:31:21.213445 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-9523bb24-91df-4ab9-b86f-60663f1f5331 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '178730f5589f472da96cc280c8b80c9c', 'request_id': 'req-9523bb24-91df-4ab9-b86f-60663f1f5331'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:31:21.229802 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-9523bb24-91df-4ab9-b86f-60663f1f5331 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '178730f5589f472da96cc280c8b80c9c', 'request_id': 'req-9523bb24-91df-4ab9-b86f-60663f1f5331'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:31:21.323606 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-9523bb24-91df-4ab9-b86f-60663f1f5331 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '178730f5589f472da96cc280c8b80c9c', 'request_id': 'req-9523bb24-91df-4ab9-b86f-60663f1f5331'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:31:21.347726 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-9523bb24-91df-4ab9-b86f-60663f1f5331 admin admin] The token cache is being invalidate because project 178730f5589f472da96cc280c8b80c9c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80539) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 12:31:21.348104 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-9523bb24-91df-4ab9-b86f-60663f1f5331 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:31:21.349214 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-9523bb24-91df-4ab9-b86f-60663f1f5331 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:31:21.351766 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1372/2761] 10.222.0.23 () {58 vars in 1232 bytes} [Thu May 19 12:31:20 2022] DELETE /identity/v3/projects/178730f5589f472da96cc280c8b80c9c => generated 0 bytes in 519 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 12:31:21.358660 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c195ddfc-9817-401b-8e36-9595af735f45 None None] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:31:21.359047 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c195ddfc-9817-401b-8e36-9595af735f45 None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:31:21.359384 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c195ddfc-9817-401b-8e36-9595af735f45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:31:21.382192 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.auth.core [None req-c195ddfc-9817-401b-8e36-9595af735f45 None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:31:21.383526 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-c195ddfc-9817-401b-8e36-9595af735f45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:31:21.463610 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1390/2762] 10.222.0.23 () {58 vars in 957 bytes} [Thu May 19 12:31:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 12:31:21.473729 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2990ad-e0bd-4c36-b5e3-fe203b4d76fb None None] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:31:21.473963 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2990ad-e0bd-4c36-b5e3-fe203b4d76fb None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:31:21.474137 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2990ad-e0bd-4c36-b5e3-fe203b4d76fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:31:21.504500 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.auth.core [None req-1c2990ad-e0bd-4c36-b5e3-fe203b4d76fb None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:31:21.505797 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-1c2990ad-e0bd-4c36-b5e3-fe203b4d76fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:31:21.524446 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1373/2763] 10.222.0.23 () {58 vars in 957 bytes} [Thu May 19 12:31:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:31:21.532284 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c195ddfc-9817-401b-8e36-9595af735f45 None None] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:31:21.539864 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c195ddfc-9817-401b-8e36-9595af735f45 None None] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:31:21.545122 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1e298e-5819-4a9e-a396-d464be3571a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:31:21.545817 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1e298e-5819-4a9e-a396-d464be3571a3 admin admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:31:21.546102 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1e298e-5819-4a9e-a396-d464be3571a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:31:21.546387 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1e298e-5819-4a9e-a396-d464be3571a3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:31:21.547141 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b1e298e-5819-4a9e-a396-d464be3571a3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:31:21.547451 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b1e298e-5819-4a9e-a396-d464be3571a3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:31:21.549194 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b1e298e-5819-4a9e-a396-d464be3571a3 admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:31:21.573205 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2990ad-e0bd-4c36-b5e3-fe203b4d76fb None None] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:31:21.579983 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1391/2764] 10.222.0.23 () {58 vars in 1152 bytes} [Thu May 19 12:31:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:31:21.584376 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2990ad-e0bd-4c36-b5e3-fe203b4d76fb None None] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:31:21.590881 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0294e03-3005-4554-87b7-95678e4dbb6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:31:21.592223 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0294e03-3005-4554-87b7-95678e4dbb6e admin admin] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:31:21.595354 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0294e03-3005-4554-87b7-95678e4dbb6e admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:31:21.595354 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0294e03-3005-4554-87b7-95678e4dbb6e admin admin] PATH_INFO: `/v3/projects` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:31:21.595354 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0294e03-3005-4554-87b7-95678e4dbb6e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:31:21.602581 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0294e03-3005-4554-87b7-95678e4dbb6e admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:31:21.722936 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1374/2765] 10.222.0.23 () {60 vars in 1152 bytes} [Thu May 19 12:31:21 2022] POST /identity/v3/projects => generated 387 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 12:31:21.728114 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1e298e-5819-4a9e-a396-d464be3571a3 admin admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:31:21.746290 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1e298e-5819-4a9e-a396-d464be3571a3 admin admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:31:21.826476 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae1fdb4-c64f-4344-8d52-1aa8b9928f01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:31:21.827571 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae1fdb4-c64f-4344-8d52-1aa8b9928f01 admin admin] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:31:21.827918 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae1fdb4-c64f-4344-8d52-1aa8b9928f01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:31:21.828205 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae1fdb4-c64f-4344-8d52-1aa8b9928f01 admin admin] PATH_INFO: `/v3/users` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:31:21.828920 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae1fdb4-c64f-4344-8d52-1aa8b9928f01 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:31:21.830918 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae1fdb4-c64f-4344-8d52-1aa8b9928f01 admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:31:21.942936 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1392/2766] 10.222.0.23 () {60 vars in 1143 bytes} [Thu May 19 12:31:21 2022] POST /identity/v3/users => generated 294 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 12:31:21.948429 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0294e03-3005-4554-87b7-95678e4dbb6e admin admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:31:21.971229 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0294e03-3005-4554-87b7-95678e4dbb6e admin admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:31:21.981071 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a673eb-9320-4f6f-a20e-0f5ac8885deb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:31:21.982311 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a673eb-9320-4f6f-a20e-0f5ac8885deb admin admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:31:21.982863 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a673eb-9320-4f6f-a20e-0f5ac8885deb admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:31:21.983564 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a673eb-9320-4f6f-a20e-0f5ac8885deb admin admin] PATH_INFO: `/v3/roles` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:31:21.985038 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a673eb-9320-4f6f-a20e-0f5ac8885deb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:31:21.988632 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a673eb-9320-4f6f-a20e-0f5ac8885deb admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:31:22.016945 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1375/2767] 10.222.0.23 () {58 vars in 1121 bytes} [Thu May 19 12:31:21 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 12:31:22.024470 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae1fdb4-c64f-4344-8d52-1aa8b9928f01 admin admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:31:22.057853 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae1fdb4-c64f-4344-8d52-1aa8b9928f01 admin admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:31:22.092999 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7fb3f4b-311c-40ff-a087-68a6d783bcc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:31:22.138680 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fb3f4b-311c-40ff-a087-68a6d783bcc0 admin admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:31:22.138680 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fb3f4b-311c-40ff-a087-68a6d783bcc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:31:22.138680 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fb3f4b-311c-40ff-a087-68a6d783bcc0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:31:22.138680 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7fb3f4b-311c-40ff-a087-68a6d783bcc0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:31:22.138680 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7fb3f4b-311c-40ff-a087-68a6d783bcc0 admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:31:22.138680 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1393/2768] 10.222.0.23 () {58 vars in 1121 bytes} [Thu May 19 12:31:22 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 12:31:22.140333 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a673eb-9320-4f6f-a20e-0f5ac8885deb admin admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:31:22.157931 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a673eb-9320-4f6f-a20e-0f5ac8885deb admin admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:31:22.168248 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92975eec-aff1-4ee5-8bb7-10b855593179 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:31:22.170248 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92975eec-aff1-4ee5-8bb7-10b855593179 admin admin] REQUEST_METHOD: `PUT` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:31:22.170928 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92975eec-aff1-4ee5-8bb7-10b855593179 admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:31:22.171507 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92975eec-aff1-4ee5-8bb7-10b855593179 admin admin] PATH_INFO: `/v3/projects/f9e056d72ec24b0d85acbe17b606ae17/users/82b0e9a733f34c73856f4c4a125993aa/roles/259eb400cdb4439db72d14456ae9cd08` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:31:22.207141 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92975eec-aff1-4ee5-8bb7-10b855593179 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f9e056d72ec24b0d85acbe17b606ae17, user_id=82b0e9a733f34c73856f4c4a125993aa, role_id=259eb400cdb4439db72d14456ae9cd08)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:31:22.210614 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92975eec-aff1-4ee5-8bb7-10b855593179 admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:31:22.400991 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1376/2769] 10.222.0.23 () {60 vars in 1482 bytes} [Thu May 19 12:31:22 2022] PUT /identity/v3/projects/f9e056d72ec24b0d85acbe17b606ae17/users/82b0e9a733f34c73856f4c4a125993aa/roles/259eb400cdb4439db72d14456ae9cd08 => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:31:22.419855 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044b15b5-d862-4467-81ef-d8bd71ec4382 None None] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:31:22.420539 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044b15b5-d862-4467-81ef-d8bd71ec4382 None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:31:22.424766 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044b15b5-d862-4467-81ef-d8bd71ec4382 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:31:22.463432 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.auth.core [None req-044b15b5-d862-4467-81ef-d8bd71ec4382 None None] MFA Rules not processed for user `82b0e9a733f34c73856f4c4a125993aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:31:22.464996 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-044b15b5-d862-4467-81ef-d8bd71ec4382 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:31:22.575282 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1394/2770] 10.222.0.23 () {58 vars in 957 bytes} [Thu May 19 12:31:22 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 12:31:22.580636 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92975eec-aff1-4ee5-8bb7-10b855593179 admin admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:31:22.596539 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:31:22.628807 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:31:22.696513 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92975eec-aff1-4ee5-8bb7-10b855593179 admin admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:31:22.704260 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b89257-32b1-42ed-8592-be954d7c7578 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:31:22.705228 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b89257-32b1-42ed-8592-be954d7c7578 admin admin] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:31:22.705731 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b89257-32b1-42ed-8592-be954d7c7578 admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:31:22.706074 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b89257-32b1-42ed-8592-be954d7c7578 admin admin] PATH_INFO: `/v3/projects` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:31:22.706887 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b89257-32b1-42ed-8592-be954d7c7578 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:31:22.710248 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b89257-32b1-42ed-8592-be954d7c7578 admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:31:22.851578 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1377/2771] 10.222.0.23 () {60 vars in 1152 bytes} [Thu May 19 12:31:22 2022] POST /identity/v3/projects => generated 387 bytes in 271 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 12:31:22.856684 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044b15b5-d862-4467-81ef-d8bd71ec4382 None None] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:31:22.868668 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044b15b5-d862-4467-81ef-d8bd71ec4382 None None] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:31:22.958546 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141015aa-53d4-439a-99e4-654d44017871 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:31:22.959399 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141015aa-53d4-439a-99e4-654d44017871 admin admin] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:31:22.959754 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141015aa-53d4-439a-99e4-654d44017871 admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:31:22.960127 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141015aa-53d4-439a-99e4-654d44017871 admin admin] PATH_INFO: `/v3/users` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:31:22.960981 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141015aa-53d4-439a-99e4-654d44017871 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:31:22.963599 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141015aa-53d4-439a-99e4-654d44017871 admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:31:23.112434 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1395/2772] 10.222.0.23 () {60 vars in 1143 bytes} [Thu May 19 12:31:22 2022] POST /identity/v3/users => generated 300 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 12:31:23.116655 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b89257-32b1-42ed-8592-be954d7c7578 admin admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:31:23.128374 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b89257-32b1-42ed-8592-be954d7c7578 admin admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:31:23.133554 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a748695a-7c2b-4676-88fb-76257786c103 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:31:23.134279 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a748695a-7c2b-4676-88fb-76257786c103 admin admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:31:23.135061 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a748695a-7c2b-4676-88fb-76257786c103 admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:31:23.135608 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a748695a-7c2b-4676-88fb-76257786c103 admin admin] PATH_INFO: `/v3/roles` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:31:23.137071 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a748695a-7c2b-4676-88fb-76257786c103 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:31:23.138861 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a748695a-7c2b-4676-88fb-76257786c103 admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:31:23.150978 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1378/2773] 10.222.0.23 () {58 vars in 1121 bytes} [Thu May 19 12:31:23 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 12:31:23.156054 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141015aa-53d4-439a-99e4-654d44017871 admin admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:31:23.167949 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141015aa-53d4-439a-99e4-654d44017871 admin admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:31:23.178629 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abc1f93-a53e-4987-b91c-afa171c28d4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:31:23.179558 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abc1f93-a53e-4987-b91c-afa171c28d4b admin admin] REQUEST_METHOD: `PUT` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:31:23.179877 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abc1f93-a53e-4987-b91c-afa171c28d4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:31:23.180166 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abc1f93-a53e-4987-b91c-afa171c28d4b admin admin] PATH_INFO: `/v3/projects/97fff9b1b26f4104ad1863247ebb5a82/users/63b66309866f49ed988f0a11d985525f/roles/9db5671c65d3414ea2682afa2ccf1403` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:31:23.198968 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3abc1f93-a53e-4987-b91c-afa171c28d4b admin admin] RBAC: Authorizing `identity:create_grant(project_id=97fff9b1b26f4104ad1863247ebb5a82, user_id=63b66309866f49ed988f0a11d985525f, role_id=9db5671c65d3414ea2682afa2ccf1403)` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:31:23.202219 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3abc1f93-a53e-4987-b91c-afa171c28d4b admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:31:23.312916 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1396/2774] 10.222.0.23 () {60 vars in 1482 bytes} [Thu May 19 12:31:23 2022] PUT /identity/v3/projects/97fff9b1b26f4104ad1863247ebb5a82/users/63b66309866f49ed988f0a11d985525f/roles/9db5671c65d3414ea2682afa2ccf1403 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:31:23.317121 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a748695a-7c2b-4676-88fb-76257786c103 admin admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:31:23.327688 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a748695a-7c2b-4676-88fb-76257786c103 admin admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:31:23.392618 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd92be9e-ca53-4f95-90d7-0a26b63e6014 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:31:23.393276 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd92be9e-ca53-4f95-90d7-0a26b63e6014 admin admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:31:23.393619 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd92be9e-ca53-4f95-90d7-0a26b63e6014 admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:31:23.393840 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd92be9e-ca53-4f95-90d7-0a26b63e6014 admin admin] PATH_INFO: `/v3/roles` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:31:23.394832 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd92be9e-ca53-4f95-90d7-0a26b63e6014 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:31:23.400667 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd92be9e-ca53-4f95-90d7-0a26b63e6014 admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:31:23.408599 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1379/2775] 10.222.0.23 () {58 vars in 1121 bytes} [Thu May 19 12:31:23 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 12:31:23.412881 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abc1f93-a53e-4987-b91c-afa171c28d4b admin admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:31:23.425096 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abc1f93-a53e-4987-b91c-afa171c28d4b admin admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:31:23.431678 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d479f17-a20b-4ab5-9616-ebec15bc9dff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:31:23.432626 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d479f17-a20b-4ab5-9616-ebec15bc9dff admin admin] REQUEST_METHOD: `PUT` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:31:23.432956 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d479f17-a20b-4ab5-9616-ebec15bc9dff admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:31:23.433244 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d479f17-a20b-4ab5-9616-ebec15bc9dff admin admin] PATH_INFO: `/v3/projects/97fff9b1b26f4104ad1863247ebb5a82/users/63b66309866f49ed988f0a11d985525f/roles/9db5671c65d3414ea2682afa2ccf1403` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:31:23.435453 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d479f17-a20b-4ab5-9616-ebec15bc9dff admin admin] RBAC: Authorizing `identity:create_grant(project_id=97fff9b1b26f4104ad1863247ebb5a82, user_id=63b66309866f49ed988f0a11d985525f, role_id=9db5671c65d3414ea2682afa2ccf1403)` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:31:23.438444 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d479f17-a20b-4ab5-9616-ebec15bc9dff admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:31:23.456389 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1397/2776] 10.222.0.23 () {60 vars in 1482 bytes} [Thu May 19 12:31:23 2022] PUT /identity/v3/projects/97fff9b1b26f4104ad1863247ebb5a82/users/63b66309866f49ed988f0a11d985525f/roles/9db5671c65d3414ea2682afa2ccf1403 => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:31:23.473623 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb39ecd-de27-4594-a593-693027b37fae None None] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:31:23.473926 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb39ecd-de27-4594-a593-693027b37fae None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:31:23.474218 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb39ecd-de27-4594-a593-693027b37fae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:31:23.505345 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.auth.core [None req-2eb39ecd-de27-4594-a593-693027b37fae None None] MFA Rules not processed for user `63b66309866f49ed988f0a11d985525f`. Rule list: `[]` (Enabled: `True`). {{(pid=80539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:31:23.506592 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-2eb39ecd-de27-4594-a593-693027b37fae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:31:23.585374 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1380/2777] 10.222.0.23 () {58 vars in 957 bytes} [Thu May 19 12:31:23 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:31:23.606105 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d479f17-a20b-4ab5-9616-ebec15bc9dff admin admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:31:23.608184 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-0d479f17-a20b-4ab5-9616-ebec15bc9dff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:31:23.704700 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d479f17-a20b-4ab5-9616-ebec15bc9dff admin admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:31:23.708486 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62b04a9-f06c-4396-b7a3-b1597716b020 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8800254c0e784e42a791fb9abc274d79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd70d25bf69314820b8aaecb65576978a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:31:23.709170 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62b04a9-f06c-4396-b7a3-b1597716b020 service cinder] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:31:23.709461 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62b04a9-f06c-4396-b7a3-b1597716b020 service cinder] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:31:23.709800 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62b04a9-f06c-4396-b7a3-b1597716b020 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:31:23.722988 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62b04a9-f06c-4396-b7a3-b1597716b020 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:31:23.724523 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62b04a9-f06c-4396-b7a3-b1597716b020 service cinder] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:31:23.737062 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1398/2778] 10.222.0.23 () {60 vars in 1365 bytes} [Thu May 19 12:31:23 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 12:31:23.924074 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb39ecd-de27-4594-a593-693027b37fae None None] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:31:23.930325 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:31:24.019595 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb39ecd-de27-4594-a593-693027b37fae None None] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:31:24.024285 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fbd411-4364-4287-8ee7-9fae8031e514 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:31:24.025278 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fbd411-4364-4287-8ee7-9fae8031e514 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:31:24.025569 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fbd411-4364-4287-8ee7-9fae8031e514 admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:31:24.025853 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fbd411-4364-4287-8ee7-9fae8031e514 admin admin] PATH_INFO: `/v3/users/82b0e9a733f34c73856f4c4a125993aa` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:31:24.027149 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7fbd411-4364-4287-8ee7-9fae8031e514 admin admin] RBAC: Authorizing `identity:delete_user(user_id=82b0e9a733f34c73856f4c4a125993aa)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:31:24.029271 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7fbd411-4364-4287-8ee7-9fae8031e514 admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:31:24.229162 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-f7fbd411-4364-4287-8ee7-9fae8031e514 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '82b0e9a733f34c73856f4c4a125993aa'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:31:24.248569 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-f7fbd411-4364-4287-8ee7-9fae8031e514 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '82b0e9a733f34c73856f4c4a125993aa'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:31:24.251582 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-f7fbd411-4364-4287-8ee7-9fae8031e514 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '82b0e9a733f34c73856f4c4a125993aa'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:31:24.340408 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-f7fbd411-4364-4287-8ee7-9fae8031e514 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '82b0e9a733f34c73856f4c4a125993aa'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:31:24.351466 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-f7fbd411-4364-4287-8ee7-9fae8031e514 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '82b0e9a733f34c73856f4c4a125993aa'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:31:24.361449 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1381/2779] 10.222.0.23 () {58 vars in 1223 bytes} [Thu May 19 12:31:23 2022] DELETE /identity/v3/users/82b0e9a733f34c73856f4c4a125993aa => generated 0 bytes in 439 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:31:24.368288 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab802a84-a715-43f9-94eb-9f719cb77290 None None] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:31:24.369096 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab802a84-a715-43f9-94eb-9f719cb77290 None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:31:24.370235 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab802a84-a715-43f9-94eb-9f719cb77290 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:31:24.380435 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:31:24.395055 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.auth.core [None req-ab802a84-a715-43f9-94eb-9f719cb77290 None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:31:24.396579 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-ab802a84-a715-43f9-94eb-9f719cb77290 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:31:24.474790 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1399/2780] 10.222.0.23 () {58 vars in 957 bytes} [Thu May 19 12:31:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:31:24.486290 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fbd411-4364-4287-8ee7-9fae8031e514 admin admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:31:24.488285 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-f7fbd411-4364-4287-8ee7-9fae8031e514 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:31:24.573544 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fbd411-4364-4287-8ee7-9fae8031e514 admin admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:31:24.577881 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6842e875-7922-4c8c-9d01-89c91b0c1849 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a00b3fa7497456895e5e0bf105c4fc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd70d25bf69314820b8aaecb65576978a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:31:24.578362 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6842e875-7922-4c8c-9d01-89c91b0c1849 service neutron] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:31:24.578648 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6842e875-7922-4c8c-9d01-89c91b0c1849 service neutron] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:31:24.578920 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6842e875-7922-4c8c-9d01-89c91b0c1849 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:31:24.594025 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6842e875-7922-4c8c-9d01-89c91b0c1849 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:31:24.596461 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6842e875-7922-4c8c-9d01-89c91b0c1849 service neutron] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:31:24.613202 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1382/2781] 10.222.0.23 () {60 vars in 1365 bytes} [Thu May 19 12:31:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 12:31:25.326357 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab802a84-a715-43f9-94eb-9f719cb77290 None None] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:31:25.327773 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-ab802a84-a715-43f9-94eb-9f719cb77290 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:31:25.342309 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab802a84-a715-43f9-94eb-9f719cb77290 None None] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:31:25.348540 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f500bc68-595f-4b98-a0a7-a5e8af6301ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:31:25.349980 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f500bc68-595f-4b98-a0a7-a5e8af6301ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:31:25.350490 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f500bc68-595f-4b98-a0a7-a5e8af6301ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:31:25.350952 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f500bc68-595f-4b98-a0a7-a5e8af6301ec admin admin] PATH_INFO: `/v3/projects/f9e056d72ec24b0d85acbe17b606ae17` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:31:25.353392 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f500bc68-595f-4b98-a0a7-a5e8af6301ec admin admin] RBAC: Authorizing `identity:delete_project(project_id=f9e056d72ec24b0d85acbe17b606ae17)` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:31:25.357674 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f500bc68-595f-4b98-a0a7-a5e8af6301ec admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:31:25.492007 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-f500bc68-595f-4b98-a0a7-a5e8af6301ec admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f9e056d72ec24b0d85acbe17b606ae17', 'request_id': 'req-f500bc68-595f-4b98-a0a7-a5e8af6301ec'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:31:25.499444 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-f500bc68-595f-4b98-a0a7-a5e8af6301ec admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f9e056d72ec24b0d85acbe17b606ae17', 'request_id': 'req-f500bc68-595f-4b98-a0a7-a5e8af6301ec'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:31:25.506518 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-f500bc68-595f-4b98-a0a7-a5e8af6301ec admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f9e056d72ec24b0d85acbe17b606ae17', 'request_id': 'req-f500bc68-595f-4b98-a0a7-a5e8af6301ec'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:31:25.592757 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-f500bc68-595f-4b98-a0a7-a5e8af6301ec admin admin] The token cache is being invalidate because project f9e056d72ec24b0d85acbe17b606ae17 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80540) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 12:31:25.593038 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-f500bc68-595f-4b98-a0a7-a5e8af6301ec admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:31:25.593629 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-f500bc68-595f-4b98-a0a7-a5e8af6301ec admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:31:25.595340 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1400/2782] 10.222.0.23 () {58 vars in 1232 bytes} [Thu May 19 12:31:25 2022] DELETE /identity/v3/projects/f9e056d72ec24b0d85acbe17b606ae17 => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:31:25.599572 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6842e875-7922-4c8c-9d01-89c91b0c1849 service neutron] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:31:25.618001 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6842e875-7922-4c8c-9d01-89c91b0c1849 service neutron] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:31:25.663424 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:31:25.711043 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2f49c4-3cc2-4409-b7e8-346067ab3ea4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:31:25.712171 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2f49c4-3cc2-4409-b7e8-346067ab3ea4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:31:25.712649 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2f49c4-3cc2-4409-b7e8-346067ab3ea4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:31:25.713033 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2f49c4-3cc2-4409-b7e8-346067ab3ea4 admin admin] PATH_INFO: `/v3/users/63b66309866f49ed988f0a11d985525f` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:31:25.714861 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2f49c4-3cc2-4409-b7e8-346067ab3ea4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=63b66309866f49ed988f0a11d985525f)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:31:25.717232 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2f49c4-3cc2-4409-b7e8-346067ab3ea4 admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:31:26.042677 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-2c2f49c4-3cc2-4409-b7e8-346067ab3ea4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '63b66309866f49ed988f0a11d985525f'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:31:26.068352 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-2c2f49c4-3cc2-4409-b7e8-346067ab3ea4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '63b66309866f49ed988f0a11d985525f'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:31:26.071096 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-2c2f49c4-3cc2-4409-b7e8-346067ab3ea4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '63b66309866f49ed988f0a11d985525f'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:31:26.288703 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-2c2f49c4-3cc2-4409-b7e8-346067ab3ea4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '63b66309866f49ed988f0a11d985525f'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:31:26.301004 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-2c2f49c4-3cc2-4409-b7e8-346067ab3ea4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '63b66309866f49ed988f0a11d985525f'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:31:26.310608 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1383/2783] 10.222.0.23 () {58 vars in 1223 bytes} [Thu May 19 12:31:25 2022] DELETE /identity/v3/users/63b66309866f49ed988f0a11d985525f => generated 0 bytes in 712 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:31:26.913223 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f500bc68-595f-4b98-a0a7-a5e8af6301ec admin admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:31:26.915038 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-f500bc68-595f-4b98-a0a7-a5e8af6301ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:31:26.939934 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f500bc68-595f-4b98-a0a7-a5e8af6301ec admin admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:31:27.041771 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31f3fb2-42c9-4c29-8d53-ec3dfe2697f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:31:27.042762 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31f3fb2-42c9-4c29-8d53-ec3dfe2697f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:31:27.043108 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31f3fb2-42c9-4c29-8d53-ec3dfe2697f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:31:27.043450 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31f3fb2-42c9-4c29-8d53-ec3dfe2697f7 admin admin] PATH_INFO: `/v3/projects/97fff9b1b26f4104ad1863247ebb5a82` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:31:27.045148 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31f3fb2-42c9-4c29-8d53-ec3dfe2697f7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=97fff9b1b26f4104ad1863247ebb5a82)` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:31:27.047828 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31f3fb2-42c9-4c29-8d53-ec3dfe2697f7 admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:31:27.190639 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:31:27.220348 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-b31f3fb2-42c9-4c29-8d53-ec3dfe2697f7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '97fff9b1b26f4104ad1863247ebb5a82', 'request_id': 'req-b31f3fb2-42c9-4c29-8d53-ec3dfe2697f7'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:31:27.230660 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-b31f3fb2-42c9-4c29-8d53-ec3dfe2697f7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '97fff9b1b26f4104ad1863247ebb5a82', 'request_id': 'req-b31f3fb2-42c9-4c29-8d53-ec3dfe2697f7'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:31:27.238364 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-b31f3fb2-42c9-4c29-8d53-ec3dfe2697f7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '97fff9b1b26f4104ad1863247ebb5a82', 'request_id': 'req-b31f3fb2-42c9-4c29-8d53-ec3dfe2697f7'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:31:27.366338 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-b31f3fb2-42c9-4c29-8d53-ec3dfe2697f7 admin admin] The token cache is being invalidate because project 97fff9b1b26f4104ad1863247ebb5a82 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80540) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 12:31:27.366338 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-b31f3fb2-42c9-4c29-8d53-ec3dfe2697f7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:31:27.366338 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-b31f3fb2-42c9-4c29-8d53-ec3dfe2697f7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:31:27.368568 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1401/2784] 10.222.0.23 () {58 vars in 1232 bytes} [Thu May 19 12:31:26 2022] DELETE /identity/v3/projects/97fff9b1b26f4104ad1863247ebb5a82 => generated 0 bytes in 457 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:31:27.378567 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe6894f-5119-4126-a657-4be1cc59ef71 None None] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:31:27.379087 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe6894f-5119-4126-a657-4be1cc59ef71 None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:31:27.379713 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe6894f-5119-4126-a657-4be1cc59ef71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:31:27.444601 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.auth.core [None req-8fe6894f-5119-4126-a657-4be1cc59ef71 None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:31:27.447374 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-8fe6894f-5119-4126-a657-4be1cc59ef71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:31:27.593306 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1384/2785] 10.222.0.23 () {58 vars in 957 bytes} [Thu May 19 12:31:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:31:27.599898 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e6630b-ebed-4aac-b53f-f9acda27b51e None None] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:31:27.600472 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e6630b-ebed-4aac-b53f-f9acda27b51e None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:31:27.601290 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e6630b-ebed-4aac-b53f-f9acda27b51e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:31:27.624365 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.auth.core [None req-c1e6630b-ebed-4aac-b53f-f9acda27b51e None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:31:27.625982 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-c1e6630b-ebed-4aac-b53f-f9acda27b51e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:31:27.640642 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1402/2786] 10.222.0.23 () {58 vars in 957 bytes} [Thu May 19 12:31:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:31:27.641673 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe6894f-5119-4126-a657-4be1cc59ef71 None None] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:31:27.659264 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe6894f-5119-4126-a657-4be1cc59ef71 None None] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:31:27.669319 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59db128b-1619-48cc-9de7-124b66bfa2c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:31:27.671101 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59db128b-1619-48cc-9de7-124b66bfa2c7 admin admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:31:27.671643 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59db128b-1619-48cc-9de7-124b66bfa2c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:31:27.672443 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59db128b-1619-48cc-9de7-124b66bfa2c7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:31:27.673931 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59db128b-1619-48cc-9de7-124b66bfa2c7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:31:27.674480 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59db128b-1619-48cc-9de7-124b66bfa2c7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:31:27.679189 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59db128b-1619-48cc-9de7-124b66bfa2c7 admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:31:27.716892 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1385/2787] 10.222.0.23 () {58 vars in 1152 bytes} [Thu May 19 12:31:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:31:27.724148 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e6630b-ebed-4aac-b53f-f9acda27b51e None None] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:31:27.743583 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e6630b-ebed-4aac-b53f-f9acda27b51e None None] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:31:27.762293 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43e8676-28e5-44a5-816b-48c8d193f6db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:31:27.763520 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43e8676-28e5-44a5-816b-48c8d193f6db admin admin] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:31:27.764163 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43e8676-28e5-44a5-816b-48c8d193f6db admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:31:27.764897 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43e8676-28e5-44a5-816b-48c8d193f6db admin admin] PATH_INFO: `/v3/projects` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:31:27.766354 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c43e8676-28e5-44a5-816b-48c8d193f6db admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:31:27.770372 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c43e8676-28e5-44a5-816b-48c8d193f6db admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:31:27.884039 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1403/2788] 10.222.0.23 () {60 vars in 1152 bytes} [Thu May 19 12:31:27 2022] POST /identity/v3/projects => generated 385 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 12:31:27.886418 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59db128b-1619-48cc-9de7-124b66bfa2c7 admin admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:31:27.897261 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59db128b-1619-48cc-9de7-124b66bfa2c7 admin admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:31:27.956188 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:31:27.991267 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd80f51-d6f6-49d3-af5b-3082e1a77bbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:31:27.992238 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd80f51-d6f6-49d3-af5b-3082e1a77bbc admin admin] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:31:27.992644 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd80f51-d6f6-49d3-af5b-3082e1a77bbc admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:31:27.993094 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd80f51-d6f6-49d3-af5b-3082e1a77bbc admin admin] PATH_INFO: `/v3/users` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:31:27.994535 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd80f51-d6f6-49d3-af5b-3082e1a77bbc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:31:27.996925 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd80f51-d6f6-49d3-af5b-3082e1a77bbc admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:31:28.129679 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1386/2789] 10.222.0.23 () {60 vars in 1143 bytes} [Thu May 19 12:31:27 2022] POST /identity/v3/users => generated 293 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 12:31:28.143310 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43e8676-28e5-44a5-816b-48c8d193f6db admin admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:31:28.159362 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43e8676-28e5-44a5-816b-48c8d193f6db admin admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:31:28.166237 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15fa0e2-3d0d-43d4-9e6a-c86eae23236d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:31:28.167010 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15fa0e2-3d0d-43d4-9e6a-c86eae23236d admin admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:31:28.167299 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15fa0e2-3d0d-43d4-9e6a-c86eae23236d admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:31:28.167589 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15fa0e2-3d0d-43d4-9e6a-c86eae23236d admin admin] PATH_INFO: `/v3/roles` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:31:28.168451 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15fa0e2-3d0d-43d4-9e6a-c86eae23236d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:31:28.170283 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15fa0e2-3d0d-43d4-9e6a-c86eae23236d admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:31:28.186826 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1404/2790] 10.222.0.23 () {58 vars in 1121 bytes} [Thu May 19 12:31:28 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 12:31:28.190832 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd80f51-d6f6-49d3-af5b-3082e1a77bbc admin admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:31:28.213173 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd80f51-d6f6-49d3-af5b-3082e1a77bbc admin admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:31:28.216669 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8868c5f1-0ce4-4286-8a98-b7380930bc9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:31:28.217876 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8868c5f1-0ce4-4286-8a98-b7380930bc9f admin admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:31:28.218376 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8868c5f1-0ce4-4286-8a98-b7380930bc9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:31:28.218869 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8868c5f1-0ce4-4286-8a98-b7380930bc9f admin admin] PATH_INFO: `/v3/roles` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:31:28.220321 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8868c5f1-0ce4-4286-8a98-b7380930bc9f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:31:28.224593 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8868c5f1-0ce4-4286-8a98-b7380930bc9f admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:31:28.246553 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1387/2791] 10.222.0.23 () {58 vars in 1121 bytes} [Thu May 19 12:31:28 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 12:31:28.250878 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15fa0e2-3d0d-43d4-9e6a-c86eae23236d admin admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:31:28.269095 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15fa0e2-3d0d-43d4-9e6a-c86eae23236d admin admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:31:28.277230 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e403339-af80-47e2-aff3-b6fce5c0ade1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:31:28.279389 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e403339-af80-47e2-aff3-b6fce5c0ade1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:31:28.280634 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e403339-af80-47e2-aff3-b6fce5c0ade1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:31:28.281116 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e403339-af80-47e2-aff3-b6fce5c0ade1 admin admin] PATH_INFO: `/v3/projects/671e9a075f864d8f97d81ef1ce543dc2/users/3a3f4c278c054e918a96b764a24adbbb/roles/259eb400cdb4439db72d14456ae9cd08` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:31:28.304007 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e403339-af80-47e2-aff3-b6fce5c0ade1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=671e9a075f864d8f97d81ef1ce543dc2, user_id=3a3f4c278c054e918a96b764a24adbbb, role_id=259eb400cdb4439db72d14456ae9cd08)` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:31:28.308893 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e403339-af80-47e2-aff3-b6fce5c0ade1 admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:31:28.427284 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1405/2792] 10.222.0.23 () {60 vars in 1482 bytes} [Thu May 19 12:31:28 2022] PUT /identity/v3/projects/671e9a075f864d8f97d81ef1ce543dc2/users/3a3f4c278c054e918a96b764a24adbbb/roles/259eb400cdb4439db72d14456ae9cd08 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:31:28.441202 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9c3c4d-01e2-4ed5-af67-e7c4b9a15e09 None None] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:31:28.442976 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9c3c4d-01e2-4ed5-af67-e7c4b9a15e09 None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:31:28.443589 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9c3c4d-01e2-4ed5-af67-e7c4b9a15e09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:31:28.488907 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.auth.core [None req-0c9c3c4d-01e2-4ed5-af67-e7c4b9a15e09 None None] MFA Rules not processed for user `3a3f4c278c054e918a96b764a24adbbb`. Rule list: `[]` (Enabled: `True`). {{(pid=80539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:31:28.491006 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-0c9c3c4d-01e2-4ed5-af67-e7c4b9a15e09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:31:28.581298 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1388/2793] 10.222.0.23 () {58 vars in 957 bytes} [Thu May 19 12:31:28 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:31:28.602218 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e403339-af80-47e2-aff3-b6fce5c0ade1 admin admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:31:28.604412 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-9e403339-af80-47e2-aff3-b6fce5c0ade1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:31:28.690383 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e403339-af80-47e2-aff3-b6fce5c0ade1 admin admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:31:28.694511 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c348525-0633-4b9f-a783-14ba659c76b5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8800254c0e784e42a791fb9abc274d79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd70d25bf69314820b8aaecb65576978a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:31:28.695220 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c348525-0633-4b9f-a783-14ba659c76b5 service cinder] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:31:28.695505 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c348525-0633-4b9f-a783-14ba659c76b5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:31:28.695810 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c348525-0633-4b9f-a783-14ba659c76b5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:31:28.708197 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c348525-0633-4b9f-a783-14ba659c76b5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:31:28.711284 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c348525-0633-4b9f-a783-14ba659c76b5 service cinder] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:31:28.733248 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1406/2794] 10.222.0.23 () {60 vars in 1365 bytes} [Thu May 19 12:31:28 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 12:31:30.367662 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9c3c4d-01e2-4ed5-af67-e7c4b9a15e09 None None] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:31:30.385457 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9c3c4d-01e2-4ed5-af67-e7c4b9a15e09 None None] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:31:30.392288 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc47f4c-3493-44ac-8574-7338ccdd328b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8800254c0e784e42a791fb9abc274d79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd70d25bf69314820b8aaecb65576978a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:31:30.393591 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc47f4c-3493-44ac-8574-7338ccdd328b service cinder] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:31:30.395104 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc47f4c-3493-44ac-8574-7338ccdd328b service cinder] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:31:30.395104 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc47f4c-3493-44ac-8574-7338ccdd328b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:31:30.399574 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-ccc47f4c-3493-44ac-8574-7338ccdd328b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:31:30.503681 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc47f4c-3493-44ac-8574-7338ccdd328b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:31:30.505464 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc47f4c-3493-44ac-8574-7338ccdd328b service cinder] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:31:30.518539 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1389/2795] 10.222.0.23 () {60 vars in 1365 bytes} [Thu May 19 12:31:30 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:31:33.199059 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c348525-0633-4b9f-a783-14ba659c76b5 service cinder] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:31:33.265148 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=92,keystone:DELETE=5,keystone:INSERT=3 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:31:33.270622 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c348525-0633-4b9f-a783-14ba659c76b5 service cinder] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:31:33.275287 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e20e5c6-a806-454e-9ee2-d451a3bb60f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:31:33.276365 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e20e5c6-a806-454e-9ee2-d451a3bb60f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:31:33.276654 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e20e5c6-a806-454e-9ee2-d451a3bb60f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:31:33.276931 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e20e5c6-a806-454e-9ee2-d451a3bb60f5 admin admin] PATH_INFO: `/v3/users/3a3f4c278c054e918a96b764a24adbbb` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:31:33.278229 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e20e5c6-a806-454e-9ee2-d451a3bb60f5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3a3f4c278c054e918a96b764a24adbbb)` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:31:33.280186 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e20e5c6-a806-454e-9ee2-d451a3bb60f5 admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:31:33.686145 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-8e20e5c6-a806-454e-9ee2-d451a3bb60f5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3a3f4c278c054e918a96b764a24adbbb'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:31:33.700738 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-8e20e5c6-a806-454e-9ee2-d451a3bb60f5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3a3f4c278c054e918a96b764a24adbbb'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:31:33.701496 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-8e20e5c6-a806-454e-9ee2-d451a3bb60f5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3a3f4c278c054e918a96b764a24adbbb'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:31:33.768228 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-8e20e5c6-a806-454e-9ee2-d451a3bb60f5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3a3f4c278c054e918a96b764a24adbbb'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:31:33.778349 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-8e20e5c6-a806-454e-9ee2-d451a3bb60f5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3a3f4c278c054e918a96b764a24adbbb'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:31:33.785907 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1407/2796] 10.222.0.23 () {58 vars in 1223 bytes} [Thu May 19 12:31:33 2022] DELETE /identity/v3/users/3a3f4c278c054e918a96b764a24adbbb => generated 0 bytes in 589 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:31:33.791152 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a78a3e-f769-4202-a95f-6df86c9529c4 None None] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:31:33.791745 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a78a3e-f769-4202-a95f-6df86c9529c4 None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:31:33.791974 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a78a3e-f769-4202-a95f-6df86c9529c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:31:33.820478 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.auth.core [None req-b0a78a3e-f769-4202-a95f-6df86c9529c4 None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:31:33.823516 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-b0a78a3e-f769-4202-a95f-6df86c9529c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:31:33.894454 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:31:33.950555 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1390/2797] 10.222.0.23 () {58 vars in 957 bytes} [Thu May 19 12:31:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:31:33.965691 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e20e5c6-a806-454e-9ee2-d451a3bb60f5 admin admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:31:33.967586 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-8e20e5c6-a806-454e-9ee2-d451a3bb60f5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:31:34.066428 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e20e5c6-a806-454e-9ee2-d451a3bb60f5 admin admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:31:34.073646 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8937b12c-daba-40a2-be74-9b0c4758ea80 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a00b3fa7497456895e5e0bf105c4fc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd70d25bf69314820b8aaecb65576978a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:31:34.074725 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8937b12c-daba-40a2-be74-9b0c4758ea80 service neutron] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:31:34.075220 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8937b12c-daba-40a2-be74-9b0c4758ea80 service neutron] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:31:34.075719 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8937b12c-daba-40a2-be74-9b0c4758ea80 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:31:34.094078 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8937b12c-daba-40a2-be74-9b0c4758ea80 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:31:34.097069 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8937b12c-daba-40a2-be74-9b0c4758ea80 service neutron] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:31:34.116365 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1408/2798] 10.222.0.23 () {60 vars in 1365 bytes} [Thu May 19 12:31:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:31:34.755942 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a78a3e-f769-4202-a95f-6df86c9529c4 None None] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:31:34.758547 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-b0a78a3e-f769-4202-a95f-6df86c9529c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:31:34.786490 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a78a3e-f769-4202-a95f-6df86c9529c4 None None] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:31:34.805134 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77e9070-2fdf-4ff3-b688-9fe706856f80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:31:34.805134 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77e9070-2fdf-4ff3-b688-9fe706856f80 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:31:34.805134 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77e9070-2fdf-4ff3-b688-9fe706856f80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:31:34.805134 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77e9070-2fdf-4ff3-b688-9fe706856f80 admin admin] PATH_INFO: `/v3/projects/671e9a075f864d8f97d81ef1ce543dc2` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:31:34.806464 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77e9070-2fdf-4ff3-b688-9fe706856f80 admin admin] RBAC: Authorizing `identity:delete_project(project_id=671e9a075f864d8f97d81ef1ce543dc2)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:31:34.806464 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77e9070-2fdf-4ff3-b688-9fe706856f80 admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:31:34.940100 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-d77e9070-2fdf-4ff3-b688-9fe706856f80 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '671e9a075f864d8f97d81ef1ce543dc2', 'request_id': 'req-d77e9070-2fdf-4ff3-b688-9fe706856f80'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:31:34.948794 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-d77e9070-2fdf-4ff3-b688-9fe706856f80 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '671e9a075f864d8f97d81ef1ce543dc2', 'request_id': 'req-d77e9070-2fdf-4ff3-b688-9fe706856f80'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:31:35.128434 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-d77e9070-2fdf-4ff3-b688-9fe706856f80 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '671e9a075f864d8f97d81ef1ce543dc2', 'request_id': 'req-d77e9070-2fdf-4ff3-b688-9fe706856f80'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:31:35.141638 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-d77e9070-2fdf-4ff3-b688-9fe706856f80 admin admin] The token cache is being invalidate because project 671e9a075f864d8f97d81ef1ce543dc2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80539) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 12:31:35.141927 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-d77e9070-2fdf-4ff3-b688-9fe706856f80 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:31:35.142704 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-d77e9070-2fdf-4ff3-b688-9fe706856f80 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:31:35.144408 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1391/2799] 10.222.0.23 () {58 vars in 1232 bytes} [Thu May 19 12:31:34 2022] DELETE /identity/v3/projects/671e9a075f864d8f97d81ef1ce543dc2 => generated 0 bytes in 390 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:31:35.149349 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cb166a-db28-47a9-a1b1-b86cf1192487 None None] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:31:35.150951 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cb166a-db28-47a9-a1b1-b86cf1192487 None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:31:35.151394 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cb166a-db28-47a9-a1b1-b86cf1192487 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:31:35.171642 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.auth.core [None req-76cb166a-db28-47a9-a1b1-b86cf1192487 None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:31:35.172959 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-76cb166a-db28-47a9-a1b1-b86cf1192487 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:31:35.244636 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1409/2800] 10.222.0.23 () {58 vars in 957 bytes} [Thu May 19 12:31:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:31:35.252691 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b901a2d3-9e81-4524-ae6b-35300ec12318 None None] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:31:35.252990 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b901a2d3-9e81-4524-ae6b-35300ec12318 None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:31:35.253418 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b901a2d3-9e81-4524-ae6b-35300ec12318 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:31:35.275742 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.auth.core [None req-b901a2d3-9e81-4524-ae6b-35300ec12318 None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:31:35.277036 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-b901a2d3-9e81-4524-ae6b-35300ec12318 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:31:35.285804 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1392/2801] 10.222.0.23 () {58 vars in 957 bytes} [Thu May 19 12:31:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:31:35.290041 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76cb166a-db28-47a9-a1b1-b86cf1192487 None None] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:31:35.301646 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76cb166a-db28-47a9-a1b1-b86cf1192487 None None] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:31:35.307684 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843d6c8f-1ee1-42a0-a633-44810fbed510 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:31:35.308603 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843d6c8f-1ee1-42a0-a633-44810fbed510 admin admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:31:35.314701 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843d6c8f-1ee1-42a0-a633-44810fbed510 admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:31:35.314701 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843d6c8f-1ee1-42a0-a633-44810fbed510 admin admin] PATH_INFO: `/v3/domains` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:31:35.314701 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843d6c8f-1ee1-42a0-a633-44810fbed510 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:31:35.314701 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843d6c8f-1ee1-42a0-a633-44810fbed510 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:31:35.314701 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843d6c8f-1ee1-42a0-a633-44810fbed510 admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:31:35.327738 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1410/2802] 10.222.0.23 () {58 vars in 1152 bytes} [Thu May 19 12:31:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 12:31:35.332653 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b901a2d3-9e81-4524-ae6b-35300ec12318 None None] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:31:35.343652 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b901a2d3-9e81-4524-ae6b-35300ec12318 None None] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:31:35.348618 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228f306f-fecb-462e-9e8a-0ff6483c259e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:31:35.349441 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228f306f-fecb-462e-9e8a-0ff6483c259e admin admin] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:31:35.349735 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228f306f-fecb-462e-9e8a-0ff6483c259e admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:31:35.350019 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228f306f-fecb-462e-9e8a-0ff6483c259e admin admin] PATH_INFO: `/v3/projects` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:31:35.350742 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228f306f-fecb-462e-9e8a-0ff6483c259e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:31:35.352823 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228f306f-fecb-462e-9e8a-0ff6483c259e admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:31:35.498230 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1393/2803] 10.222.0.23 () {60 vars in 1152 bytes} [Thu May 19 12:31:35 2022] POST /identity/v3/projects => generated 369 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 12:31:35.504091 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843d6c8f-1ee1-42a0-a633-44810fbed510 admin admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:31:35.512596 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:31:35.521254 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843d6c8f-1ee1-42a0-a633-44810fbed510 admin admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:31:35.764993 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a04c61-7594-4d75-bb5e-352df48cd111 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:31:35.765768 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a04c61-7594-4d75-bb5e-352df48cd111 admin admin] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:31:35.766150 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a04c61-7594-4d75-bb5e-352df48cd111 admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:31:35.766521 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a04c61-7594-4d75-bb5e-352df48cd111 admin admin] PATH_INFO: `/v3/users` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:31:35.767345 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a04c61-7594-4d75-bb5e-352df48cd111 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:31:35.769561 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a04c61-7594-4d75-bb5e-352df48cd111 admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:31:35.850342 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1411/2804] 10.222.0.23 () {60 vars in 1143 bytes} [Thu May 19 12:31:35 2022] POST /identity/v3/users => generated 285 bytes in 347 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 12:31:35.857322 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228f306f-fecb-462e-9e8a-0ff6483c259e admin admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:31:35.870019 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228f306f-fecb-462e-9e8a-0ff6483c259e admin admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:31:35.875003 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e350b78-d7f0-4ca5-bcf3-ad5d881d57e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:31:35.875766 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e350b78-d7f0-4ca5-bcf3-ad5d881d57e3 admin admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:31:35.876105 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e350b78-d7f0-4ca5-bcf3-ad5d881d57e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:31:35.876443 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e350b78-d7f0-4ca5-bcf3-ad5d881d57e3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:31:35.877257 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e350b78-d7f0-4ca5-bcf3-ad5d881d57e3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:31:35.878807 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e350b78-d7f0-4ca5-bcf3-ad5d881d57e3 admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:31:35.892824 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1394/2805] 10.222.0.23 () {58 vars in 1121 bytes} [Thu May 19 12:31:35 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 12:31:35.897179 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a04c61-7594-4d75-bb5e-352df48cd111 admin admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:31:35.908996 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a04c61-7594-4d75-bb5e-352df48cd111 admin admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:31:35.914096 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc6ba24-d6f1-49d9-bf1b-689329a636a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:31:35.914791 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc6ba24-d6f1-49d9-bf1b-689329a636a7 admin admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:31:35.915075 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc6ba24-d6f1-49d9-bf1b-689329a636a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:31:35.916939 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc6ba24-d6f1-49d9-bf1b-689329a636a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:31:35.916939 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc6ba24-d6f1-49d9-bf1b-689329a636a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:31:35.919090 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc6ba24-d6f1-49d9-bf1b-689329a636a7 admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:31:35.932137 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1412/2806] 10.222.0.23 () {58 vars in 1121 bytes} [Thu May 19 12:31:35 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 19 12:31:35.937092 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e350b78-d7f0-4ca5-bcf3-ad5d881d57e3 admin admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:31:35.955035 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e350b78-d7f0-4ca5-bcf3-ad5d881d57e3 admin admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:31:35.960394 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774fac51-b933-4f5e-89de-aec7321a2911 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:31:35.961550 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774fac51-b933-4f5e-89de-aec7321a2911 admin admin] REQUEST_METHOD: `PUT` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:31:35.961905 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774fac51-b933-4f5e-89de-aec7321a2911 admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:31:35.962281 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774fac51-b933-4f5e-89de-aec7321a2911 admin admin] PATH_INFO: `/v3/projects/753c6657bdfb409dab2c4bdd006818b8/users/5508b241f22d4208a8efe645509266ac/roles/259eb400cdb4439db72d14456ae9cd08` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:31:35.982467 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774fac51-b933-4f5e-89de-aec7321a2911 admin admin] RBAC: Authorizing `identity:create_grant(project_id=753c6657bdfb409dab2c4bdd006818b8, user_id=5508b241f22d4208a8efe645509266ac, role_id=259eb400cdb4439db72d14456ae9cd08)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:31:35.985350 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774fac51-b933-4f5e-89de-aec7321a2911 admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:31:36.119511 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1395/2807] 10.222.0.23 () {60 vars in 1482 bytes} [Thu May 19 12:31:35 2022] PUT /identity/v3/projects/753c6657bdfb409dab2c4bdd006818b8/users/5508b241f22d4208a8efe645509266ac/roles/259eb400cdb4439db72d14456ae9cd08 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:31:36.138423 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbfafb7-2123-4755-9dc8-b58834ec1293 None None] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:31:36.138846 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbfafb7-2123-4755-9dc8-b58834ec1293 None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:31:36.139170 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbfafb7-2123-4755-9dc8-b58834ec1293 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:31:36.171266 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.auth.core [None req-7bbfafb7-2123-4755-9dc8-b58834ec1293 None None] MFA Rules not processed for user `5508b241f22d4208a8efe645509266ac`. Rule list: `[]` (Enabled: `True`). {{(pid=80540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:31:36.172562 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-7bbfafb7-2123-4755-9dc8-b58834ec1293 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:31:36.277470 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1413/2808] 10.222.0.23 () {58 vars in 957 bytes} [Thu May 19 12:31:36 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 12:31:36.289886 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774fac51-b933-4f5e-89de-aec7321a2911 admin admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:31:36.292564 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-774fac51-b933-4f5e-89de-aec7321a2911 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:31:36.363712 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774fac51-b933-4f5e-89de-aec7321a2911 admin admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:31:36.370572 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b893c236-786c-4ff0-b04a-21c6ba85ae48 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '21697802f2be42f9baa8e07490d5ebdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd70d25bf69314820b8aaecb65576978a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:31:36.371686 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b893c236-786c-4ff0-b04a-21c6ba85ae48 service glance] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:31:36.372235 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b893c236-786c-4ff0-b04a-21c6ba85ae48 service glance] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:31:36.372735 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b893c236-786c-4ff0-b04a-21c6ba85ae48 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:31:36.389378 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b893c236-786c-4ff0-b04a-21c6ba85ae48 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:31:36.392565 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b893c236-786c-4ff0-b04a-21c6ba85ae48 service glance] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:31:36.399435 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:31:36.411008 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1396/2809] 10.222.0.23 () {60 vars in 1365 bytes} [Thu May 19 12:31:36 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:31:37.263082 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccc9edc-9280-43e1-9b11-57a1c0532971 None None] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:31:37.263660 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccc9edc-9280-43e1-9b11-57a1c0532971 None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:31:37.263822 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccc9edc-9280-43e1-9b11-57a1c0532971 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:31:37.359754 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.auth.core [None req-fccc9edc-9280-43e1-9b11-57a1c0532971 None None] MFA Rules not processed for user `0c3181fb4f584a549a786adfefdf3dfb`. Rule list: `[]` (Enabled: `True`). {{(pid=80540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:31:37.361090 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-fccc9edc-9280-43e1-9b11-57a1c0532971 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:31:37.402984 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:31:37.440478 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1414/2810] 10.222.0.23 () {58 vars in 1009 bytes} [Thu May 19 12:31:37 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:31:37.648968 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b893c236-786c-4ff0-b04a-21c6ba85ae48 service glance] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:31:37.650645 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-b893c236-786c-4ff0-b04a-21c6ba85ae48 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:31:37.815131 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b893c236-786c-4ff0-b04a-21c6ba85ae48 service glance] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:31:37.825653 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eae1bd1-6cf7-4ebd-9908-1a7f62c1bfff service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15cbfc73ae1243d2a8b1e70f9cd99876', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd70d25bf69314820b8aaecb65576978a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:31:37.826752 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eae1bd1-6cf7-4ebd-9908-1a7f62c1bfff service swift] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:31:37.827229 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eae1bd1-6cf7-4ebd-9908-1a7f62c1bfff service swift] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:31:37.827723 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eae1bd1-6cf7-4ebd-9908-1a7f62c1bfff service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:31:37.842222 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eae1bd1-6cf7-4ebd-9908-1a7f62c1bfff service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:31:37.843993 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eae1bd1-6cf7-4ebd-9908-1a7f62c1bfff service swift] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:31:37.854184 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1397/2811] 10.222.0.23 () {60 vars in 1384 bytes} [Thu May 19 12:31:37 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 206 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 19 12:31:37.992154 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fccc9edc-9280-43e1-9b11-57a1c0532971 None None] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:31:37.994611 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-fccc9edc-9280-43e1-9b11-57a1c0532971 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:31:38.090031 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fccc9edc-9280-43e1-9b11-57a1c0532971 None None] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:31:38.094966 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ab4b16-4979-40ea-86cb-bfeca266c26c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:31:38.096667 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ab4b16-4979-40ea-86cb-bfeca266c26c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:31:38.096667 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ab4b16-4979-40ea-86cb-bfeca266c26c admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:31:38.096901 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ab4b16-4979-40ea-86cb-bfeca266c26c admin admin] PATH_INFO: `/v3/users/607c1b5e793647b78fb4f7a9222ab5c8` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:31:38.098304 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ab4b16-4979-40ea-86cb-bfeca266c26c admin admin] RBAC: Authorizing `identity:delete_user(user_id=607c1b5e793647b78fb4f7a9222ab5c8)` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:31:38.100638 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ab4b16-4979-40ea-86cb-bfeca266c26c admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:31:38.322260 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-38ab4b16-4979-40ea-86cb-bfeca266c26c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '607c1b5e793647b78fb4f7a9222ab5c8'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:31:38.336371 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-38ab4b16-4979-40ea-86cb-bfeca266c26c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '607c1b5e793647b78fb4f7a9222ab5c8'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:31:38.337696 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-38ab4b16-4979-40ea-86cb-bfeca266c26c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '607c1b5e793647b78fb4f7a9222ab5c8'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:31:38.442992 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-38ab4b16-4979-40ea-86cb-bfeca266c26c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '607c1b5e793647b78fb4f7a9222ab5c8'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:31:38.456316 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-38ab4b16-4979-40ea-86cb-bfeca266c26c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '607c1b5e793647b78fb4f7a9222ab5c8'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:31:38.464919 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1415/2812] 10.222.0.23 () {58 vars in 1223 bytes} [Thu May 19 12:31:37 2022] DELETE /identity/v3/users/607c1b5e793647b78fb4f7a9222ab5c8 => generated 0 bytes in 474 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:31:38.471271 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd2943c-8210-4fc0-8c10-85c29ea70d63 None None] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:31:38.480001 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd2943c-8210-4fc0-8c10-85c29ea70d63 None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:31:38.480001 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd2943c-8210-4fc0-8c10-85c29ea70d63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:31:38.496343 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.auth.core [None req-0bd2943c-8210-4fc0-8c10-85c29ea70d63 None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:31:38.497632 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-0bd2943c-8210-4fc0-8c10-85c29ea70d63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:31:38.597572 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1398/2813] 10.222.0.23 () {58 vars in 957 bytes} [Thu May 19 12:31:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:31:38.611521 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ab4b16-4979-40ea-86cb-bfeca266c26c admin admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:31:38.614208 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-38ab4b16-4979-40ea-86cb-bfeca266c26c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:31:38.714379 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ab4b16-4979-40ea-86cb-bfeca266c26c admin admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:31:38.725567 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4536722-7030-4e3c-9e8c-19cd27a0469c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a00b3fa7497456895e5e0bf105c4fc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd70d25bf69314820b8aaecb65576978a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:31:38.727144 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4536722-7030-4e3c-9e8c-19cd27a0469c service neutron] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:31:38.727647 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4536722-7030-4e3c-9e8c-19cd27a0469c service neutron] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:31:38.728225 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4536722-7030-4e3c-9e8c-19cd27a0469c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:31:38.744848 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4536722-7030-4e3c-9e8c-19cd27a0469c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:31:38.746670 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4536722-7030-4e3c-9e8c-19cd27a0469c service neutron] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:31:38.751507 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:31:38.758715 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1416/2814] 10.222.0.23 () {60 vars in 1365 bytes} [Thu May 19 12:31:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:31:39.036996 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d0c40a-5d80-44b6-b721-8ae89f02327c None None] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:31:39.037697 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d0c40a-5d80-44b6-b721-8ae89f02327c None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:31:39.038664 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d0c40a-5d80-44b6-b721-8ae89f02327c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:31:39.065505 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.auth.core [None req-52d0c40a-5d80-44b6-b721-8ae89f02327c None None] MFA Rules not processed for user `0c3181fb4f584a549a786adfefdf3dfb`. Rule list: `[]` (Enabled: `True`). {{(pid=80539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:31:39.067548 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-52d0c40a-5d80-44b6-b721-8ae89f02327c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:31:39.155409 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1399/2815] 10.222.0.23 () {58 vars in 967 bytes} [Thu May 19 12:31:39 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:31:39.169941 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4536722-7030-4e3c-9e8c-19cd27a0469c service neutron] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:31:39.171412 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-b4536722-7030-4e3c-9e8c-19cd27a0469c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:31:39.281215 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4536722-7030-4e3c-9e8c-19cd27a0469c service neutron] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:31:39.287349 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d1f18c-0961-45ed-a57b-3b9d4c92f845 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15cbfc73ae1243d2a8b1e70f9cd99876', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd70d25bf69314820b8aaecb65576978a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:31:39.288060 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d1f18c-0961-45ed-a57b-3b9d4c92f845 service swift] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:31:39.288341 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d1f18c-0961-45ed-a57b-3b9d4c92f845 service swift] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:31:39.288622 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d1f18c-0961-45ed-a57b-3b9d4c92f845 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:31:39.299407 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d1f18c-0961-45ed-a57b-3b9d4c92f845 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:31:39.301296 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d1f18c-0961-45ed-a57b-3b9d4c92f845 service swift] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:31:39.315709 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1417/2816] 10.222.0.23 () {60 vars in 1384 bytes} [Thu May 19 12:31:39 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 19 12:31:39.399985 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d0c40a-5d80-44b6-b721-8ae89f02327c None None] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:31:39.401892 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-52d0c40a-5d80-44b6-b721-8ae89f02327c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:31:39.426563 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d0c40a-5d80-44b6-b721-8ae89f02327c None None] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:31:39.439060 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fcccbb-466f-496f-967f-02ce2cf256f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:31:39.439958 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fcccbb-466f-496f-967f-02ce2cf256f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:31:39.440600 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fcccbb-466f-496f-967f-02ce2cf256f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:31:39.441651 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fcccbb-466f-496f-967f-02ce2cf256f4 admin admin] PATH_INFO: `/v3/projects/f386834878664344a889044da5df964a` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:31:39.443995 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58fcccbb-466f-496f-967f-02ce2cf256f4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f386834878664344a889044da5df964a)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:31:39.450636 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58fcccbb-466f-496f-967f-02ce2cf256f4 admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:31:39.466427 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=100 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:31:39.613760 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-58fcccbb-466f-496f-967f-02ce2cf256f4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f386834878664344a889044da5df964a', 'request_id': 'req-58fcccbb-466f-496f-967f-02ce2cf256f4'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:31:39.621927 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-58fcccbb-466f-496f-967f-02ce2cf256f4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f386834878664344a889044da5df964a', 'request_id': 'req-58fcccbb-466f-496f-967f-02ce2cf256f4'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:31:39.716174 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-58fcccbb-466f-496f-967f-02ce2cf256f4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f386834878664344a889044da5df964a', 'request_id': 'req-58fcccbb-466f-496f-967f-02ce2cf256f4'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:31:39.731403 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-58fcccbb-466f-496f-967f-02ce2cf256f4 admin admin] The token cache is being invalidate because project f386834878664344a889044da5df964a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80539) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 12:31:39.731709 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-58fcccbb-466f-496f-967f-02ce2cf256f4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:31:39.732451 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-58fcccbb-466f-496f-967f-02ce2cf256f4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:31:39.734180 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1400/2817] 10.222.0.23 () {58 vars in 1232 bytes} [Thu May 19 12:31:39 2022] DELETE /identity/v3/projects/f386834878664344a889044da5df964a => generated 0 bytes in 335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:31:39.739052 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d1f18c-0961-45ed-a57b-3b9d4c92f845 service swift] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:31:39.757912 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d1f18c-0961-45ed-a57b-3b9d4c92f845 service swift] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:31:39.879507 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0647ccf6-cc38-4e5e-bc36-872fbd928976 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:31:39.881521 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0647ccf6-cc38-4e5e-bc36-872fbd928976 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:31:39.881750 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0647ccf6-cc38-4e5e-bc36-872fbd928976 admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:31:39.882020 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0647ccf6-cc38-4e5e-bc36-872fbd928976 admin admin] PATH_INFO: `/v3/users/f2c58ab7ebd64355afd6bd07ba188531` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:31:39.883965 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0647ccf6-cc38-4e5e-bc36-872fbd928976 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f2c58ab7ebd64355afd6bd07ba188531)` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:31:39.887647 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0647ccf6-cc38-4e5e-bc36-872fbd928976 admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:31:40.082143 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed96721c-efb8-4a1a-9c8c-271c64210093 None None] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:31:40.082985 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed96721c-efb8-4a1a-9c8c-271c64210093 None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:31:40.083198 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed96721c-efb8-4a1a-9c8c-271c64210093 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:31:40.103500 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.auth.core [None req-ed96721c-efb8-4a1a-9c8c-271c64210093 None None] MFA Rules not processed for user `0c3181fb4f584a549a786adfefdf3dfb`. Rule list: `[]` (Enabled: `True`). {{(pid=80539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:31:40.105015 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-ed96721c-efb8-4a1a-9c8c-271c64210093 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:31:40.184369 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1401/2818] 10.222.0.23 () {58 vars in 1009 bytes} [Thu May 19 12:31:40 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 12:31:40.201060 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed96721c-efb8-4a1a-9c8c-271c64210093 None None] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:31:40.203013 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-ed96721c-efb8-4a1a-9c8c-271c64210093 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:31:40.205398 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-0647ccf6-cc38-4e5e-bc36-872fbd928976 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f2c58ab7ebd64355afd6bd07ba188531'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:31:40.226159 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-0647ccf6-cc38-4e5e-bc36-872fbd928976 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f2c58ab7ebd64355afd6bd07ba188531'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:31:40.227171 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-0647ccf6-cc38-4e5e-bc36-872fbd928976 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f2c58ab7ebd64355afd6bd07ba188531'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:31:40.322550 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed96721c-efb8-4a1a-9c8c-271c64210093 None None] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:31:40.331403 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5c727e-d28e-430a-ae62-de8d03e52abc service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15cbfc73ae1243d2a8b1e70f9cd99876', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd70d25bf69314820b8aaecb65576978a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:31:40.332729 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5c727e-d28e-430a-ae62-de8d03e52abc service swift] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:31:40.333237 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5c727e-d28e-430a-ae62-de8d03e52abc service swift] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:31:40.333731 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5c727e-d28e-430a-ae62-de8d03e52abc service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:31:40.345210 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-0647ccf6-cc38-4e5e-bc36-872fbd928976 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f2c58ab7ebd64355afd6bd07ba188531'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:31:40.348849 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5c727e-d28e-430a-ae62-de8d03e52abc service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:31:40.350696 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5c727e-d28e-430a-ae62-de8d03e52abc service swift] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:31:40.357310 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-0647ccf6-cc38-4e5e-bc36-872fbd928976 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f2c58ab7ebd64355afd6bd07ba188531'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:31:40.361877 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1402/2819] 10.222.0.23 () {60 vars in 1384 bytes} [Thu May 19 12:31:40 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) May 19 12:31:40.366929 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1418/2820] 10.222.0.23 () {58 vars in 1223 bytes} [Thu May 19 12:31:39 2022] DELETE /identity/v3/users/f2c58ab7ebd64355afd6bd07ba188531 => generated 0 bytes in 629 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:31:41.120172 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5c727e-d28e-430a-ae62-de8d03e52abc service swift] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:31:41.122329 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-fa5c727e-d28e-430a-ae62-de8d03e52abc service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:31:41.148395 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5c727e-d28e-430a-ae62-de8d03e52abc service swift] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:31:41.206569 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff5d769-41d3-416a-81ae-9d7707eb91b5 None None] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:31:41.208090 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff5d769-41d3-416a-81ae-9d7707eb91b5 None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:31:41.208090 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff5d769-41d3-416a-81ae-9d7707eb91b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:31:41.216732 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:31:41.223582 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:31:41.230786 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.auth.core [None req-8ff5d769-41d3-416a-81ae-9d7707eb91b5 None None] MFA Rules not processed for user `0c3181fb4f584a549a786adfefdf3dfb`. Rule list: `[]` (Enabled: `True`). {{(pid=80540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:31:41.232076 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-8ff5d769-41d3-416a-81ae-9d7707eb91b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:31:41.254414 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de12a50-38dd-42f0-acef-6a45e585d0c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:31:41.255418 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de12a50-38dd-42f0-acef-6a45e585d0c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:31:41.255701 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de12a50-38dd-42f0-acef-6a45e585d0c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:31:41.256045 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de12a50-38dd-42f0-acef-6a45e585d0c0 admin admin] PATH_INFO: `/v3/projects/36bd679d2cc84ead85750d030d91df00` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:31:41.257369 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de12a50-38dd-42f0-acef-6a45e585d0c0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=36bd679d2cc84ead85750d030d91df00)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:31:41.260510 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de12a50-38dd-42f0-acef-6a45e585d0c0 admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:31:41.317966 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1419/2821] 10.222.0.23 () {58 vars in 967 bytes} [Thu May 19 12:31:41 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 12:31:41.332124 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff5d769-41d3-416a-81ae-9d7707eb91b5 None None] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:31:41.334013 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-8ff5d769-41d3-416a-81ae-9d7707eb91b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:31:41.420559 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff5d769-41d3-416a-81ae-9d7707eb91b5 None None] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:31:41.427476 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aedcfe4c-b5c1-4e60-9512-0a339a05ffc2 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15cbfc73ae1243d2a8b1e70f9cd99876', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd70d25bf69314820b8aaecb65576978a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:31:41.428398 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedcfe4c-b5c1-4e60-9512-0a339a05ffc2 service swift] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:31:41.428987 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedcfe4c-b5c1-4e60-9512-0a339a05ffc2 service swift] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:31:41.429510 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedcfe4c-b5c1-4e60-9512-0a339a05ffc2 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:31:41.441729 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aedcfe4c-b5c1-4e60-9512-0a339a05ffc2 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:31:41.443650 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aedcfe4c-b5c1-4e60-9512-0a339a05ffc2 service swift] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:31:41.446715 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-4de12a50-38dd-42f0-acef-6a45e585d0c0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '36bd679d2cc84ead85750d030d91df00', 'request_id': 'req-4de12a50-38dd-42f0-acef-6a45e585d0c0'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:31:41.454701 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1420/2822] 10.222.0.23 () {60 vars in 1384 bytes} [Thu May 19 12:31:41 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) May 19 12:31:41.459359 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-4de12a50-38dd-42f0-acef-6a45e585d0c0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '36bd679d2cc84ead85750d030d91df00', 'request_id': 'req-4de12a50-38dd-42f0-acef-6a45e585d0c0'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:31:41.740110 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-4de12a50-38dd-42f0-acef-6a45e585d0c0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '36bd679d2cc84ead85750d030d91df00', 'request_id': 'req-4de12a50-38dd-42f0-acef-6a45e585d0c0'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:31:41.755394 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-4de12a50-38dd-42f0-acef-6a45e585d0c0 admin admin] The token cache is being invalidate because project 36bd679d2cc84ead85750d030d91df00 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80539) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 12:31:41.755860 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-4de12a50-38dd-42f0-acef-6a45e585d0c0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:31:41.759961 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-4de12a50-38dd-42f0-acef-6a45e585d0c0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:31:41.759961 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1403/2823] 10.222.0.23 () {58 vars in 1232 bytes} [Thu May 19 12:31:41 2022] DELETE /identity/v3/projects/36bd679d2cc84ead85750d030d91df00 => generated 0 bytes in 640 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 12:31:41.765999 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e718fa-8360-4cb9-bd8b-f17d537c0a2c None None] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:31:41.766499 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e718fa-8360-4cb9-bd8b-f17d537c0a2c None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:31:41.766934 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e718fa-8360-4cb9-bd8b-f17d537c0a2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:31:41.786614 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.auth.core [None req-77e718fa-8360-4cb9-bd8b-f17d537c0a2c None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:31:41.787907 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-77e718fa-8360-4cb9-bd8b-f17d537c0a2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:31:41.865810 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1421/2824] 10.222.0.23 () {58 vars in 957 bytes} [Thu May 19 12:31:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 12:31:41.882301 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a41fc4-37fb-4e38-bf93-06d659967d02 None None] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:31:41.882607 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a41fc4-37fb-4e38-bf93-06d659967d02 None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:31:41.882889 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a41fc4-37fb-4e38-bf93-06d659967d02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:31:41.919700 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.auth.core [None req-31a41fc4-37fb-4e38-bf93-06d659967d02 None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:31:41.921135 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-31a41fc4-37fb-4e38-bf93-06d659967d02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:31:41.930464 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1404/2825] 10.222.0.23 () {58 vars in 957 bytes} [Thu May 19 12:31:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:31:41.937770 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e718fa-8360-4cb9-bd8b-f17d537c0a2c None None] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:31:41.954451 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e718fa-8360-4cb9-bd8b-f17d537c0a2c None None] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:31:41.961800 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda1eea2-f951-4257-a192-28e4986d72f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:31:41.963765 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda1eea2-f951-4257-a192-28e4986d72f3 admin admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:31:41.964299 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda1eea2-f951-4257-a192-28e4986d72f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:31:41.964798 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda1eea2-f951-4257-a192-28e4986d72f3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:31:41.966245 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda1eea2-f951-4257-a192-28e4986d72f3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80540) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:31:41.966739 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda1eea2-f951-4257-a192-28e4986d72f3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:31:41.968693 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda1eea2-f951-4257-a192-28e4986d72f3 admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:31:41.983321 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1422/2826] 10.222.0.23 () {58 vars in 1152 bytes} [Thu May 19 12:31:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:31:41.990760 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a41fc4-37fb-4e38-bf93-06d659967d02 None None] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:31:42.001881 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a41fc4-37fb-4e38-bf93-06d659967d02 None None] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:31:42.007309 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4663a1eb-2e36-43be-8175-a8432f3e7e40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:31:42.008520 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4663a1eb-2e36-43be-8175-a8432f3e7e40 admin admin] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:31:42.009601 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4663a1eb-2e36-43be-8175-a8432f3e7e40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:31:42.010099 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4663a1eb-2e36-43be-8175-a8432f3e7e40 admin admin] PATH_INFO: `/v3/projects` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:31:42.010906 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4663a1eb-2e36-43be-8175-a8432f3e7e40 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:31:42.013235 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4663a1eb-2e36-43be-8175-a8432f3e7e40 admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:31:42.164255 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1405/2827] 10.222.0.23 () {60 vars in 1152 bytes} [Thu May 19 12:31:41 2022] POST /identity/v3/projects => generated 399 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 12:31:42.170151 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda1eea2-f951-4257-a192-28e4986d72f3 admin admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:31:42.186959 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda1eea2-f951-4257-a192-28e4986d72f3 admin admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:31:42.232637 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=100 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:31:42.291854 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e2672f-fef3-4766-bce6-dd40c4bcac5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:31:42.293302 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e2672f-fef3-4766-bce6-dd40c4bcac5c admin admin] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:31:42.293870 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e2672f-fef3-4766-bce6-dd40c4bcac5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:31:42.294782 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e2672f-fef3-4766-bce6-dd40c4bcac5c admin admin] PATH_INFO: `/v3/users` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:31:42.295950 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e2672f-fef3-4766-bce6-dd40c4bcac5c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:31:42.300168 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e2672f-fef3-4766-bce6-dd40c4bcac5c admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:31:42.416257 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1423/2828] 10.222.0.23 () {60 vars in 1143 bytes} [Thu May 19 12:31:42 2022] POST /identity/v3/users => generated 300 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 12:31:42.423227 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4663a1eb-2e36-43be-8175-a8432f3e7e40 admin admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:31:42.443384 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4663a1eb-2e36-43be-8175-a8432f3e7e40 admin admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:31:42.452402 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5568b727-0881-49f5-9b39-7477cfa4954d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:31:42.453499 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5568b727-0881-49f5-9b39-7477cfa4954d admin admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:31:42.453964 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5568b727-0881-49f5-9b39-7477cfa4954d admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:31:42.454420 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5568b727-0881-49f5-9b39-7477cfa4954d admin admin] PATH_INFO: `/v3/roles` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:31:42.455721 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5568b727-0881-49f5-9b39-7477cfa4954d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:31:42.458741 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5568b727-0881-49f5-9b39-7477cfa4954d admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:31:42.474197 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1406/2829] 10.222.0.23 () {58 vars in 1121 bytes} [Thu May 19 12:31:42 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 12:31:42.480089 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e2672f-fef3-4766-bce6-dd40c4bcac5c admin admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:31:42.496689 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e2672f-fef3-4766-bce6-dd40c4bcac5c admin admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:31:42.502528 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a44cb3a-675c-4eb5-8c5b-faa8e362a65a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:31:42.503139 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a44cb3a-675c-4eb5-8c5b-faa8e362a65a admin admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:31:42.503587 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a44cb3a-675c-4eb5-8c5b-faa8e362a65a admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:31:42.503973 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a44cb3a-675c-4eb5-8c5b-faa8e362a65a admin admin] PATH_INFO: `/v3/roles` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:31:42.504914 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a44cb3a-675c-4eb5-8c5b-faa8e362a65a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:31:42.506582 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a44cb3a-675c-4eb5-8c5b-faa8e362a65a admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:31:42.520137 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1424/2830] 10.222.0.23 () {58 vars in 1121 bytes} [Thu May 19 12:31:42 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 19 12:31:42.525485 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5568b727-0881-49f5-9b39-7477cfa4954d admin admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:31:42.537170 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5568b727-0881-49f5-9b39-7477cfa4954d admin admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:31:42.542772 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80841032-26bb-4d2d-a08e-cbb3b9719889 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:31:42.543684 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80841032-26bb-4d2d-a08e-cbb3b9719889 admin admin] REQUEST_METHOD: `PUT` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:31:42.544274 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80841032-26bb-4d2d-a08e-cbb3b9719889 admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:31:42.544583 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80841032-26bb-4d2d-a08e-cbb3b9719889 admin admin] PATH_INFO: `/v3/projects/c939c19350bf409f9e11612020da5b9f/users/5a22b181786d47369715c98416f7bae8/roles/259eb400cdb4439db72d14456ae9cd08` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:31:42.567395 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80841032-26bb-4d2d-a08e-cbb3b9719889 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c939c19350bf409f9e11612020da5b9f, user_id=5a22b181786d47369715c98416f7bae8, role_id=259eb400cdb4439db72d14456ae9cd08)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:31:42.570206 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80841032-26bb-4d2d-a08e-cbb3b9719889 admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:31:42.685173 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1407/2831] 10.222.0.23 () {60 vars in 1482 bytes} [Thu May 19 12:31:42 2022] PUT /identity/v3/projects/c939c19350bf409f9e11612020da5b9f/users/5a22b181786d47369715c98416f7bae8/roles/259eb400cdb4439db72d14456ae9cd08 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:31:42.705340 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68904a41-92f6-4b88-8748-328aa3c0e890 None None] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:31:42.707165 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68904a41-92f6-4b88-8748-328aa3c0e890 None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:31:42.709633 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68904a41-92f6-4b88-8748-328aa3c0e890 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:31:42.766507 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.auth.core [None req-68904a41-92f6-4b88-8748-328aa3c0e890 None None] MFA Rules not processed for user `5a22b181786d47369715c98416f7bae8`. Rule list: `[]` (Enabled: `True`). {{(pid=80540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:31:42.767999 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-68904a41-92f6-4b88-8748-328aa3c0e890 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:31:42.848049 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1425/2832] 10.222.0.23 () {58 vars in 957 bytes} [Thu May 19 12:31:42 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:31:42.866218 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80841032-26bb-4d2d-a08e-cbb3b9719889 admin admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:31:42.868274 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-80841032-26bb-4d2d-a08e-cbb3b9719889 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:31:42.938319 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80841032-26bb-4d2d-a08e-cbb3b9719889 admin admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:31:42.944498 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eef8005-b111-4cd6-8739-a95ed92bc964 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8800254c0e784e42a791fb9abc274d79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd70d25bf69314820b8aaecb65576978a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:31:42.945685 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eef8005-b111-4cd6-8739-a95ed92bc964 service cinder] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:31:42.946199 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eef8005-b111-4cd6-8739-a95ed92bc964 service cinder] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:31:42.946758 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eef8005-b111-4cd6-8739-a95ed92bc964 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:31:42.958174 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eef8005-b111-4cd6-8739-a95ed92bc964 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:31:42.959689 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eef8005-b111-4cd6-8739-a95ed92bc964 service cinder] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:31:42.965799 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:31:42.971154 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1408/2833] 10.222.0.23 () {60 vars in 1365 bytes} [Thu May 19 12:31:42 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:31:44.956457 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68904a41-92f6-4b88-8748-328aa3c0e890 None None] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:31:44.957430 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-68904a41-92f6-4b88-8748-328aa3c0e890 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:31:45.036086 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68904a41-92f6-4b88-8748-328aa3c0e890 None None] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:31:45.043239 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740412b5-e09a-4f7d-9afa-cdd5b5fece45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:31:45.044188 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740412b5-e09a-4f7d-9afa-cdd5b5fece45 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:31:45.044481 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740412b5-e09a-4f7d-9afa-cdd5b5fece45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:31:45.044761 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740412b5-e09a-4f7d-9afa-cdd5b5fece45 admin admin] PATH_INFO: `/v3/users/d9922026c1784ed4bd40bbb5c139cf04` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:31:45.046063 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740412b5-e09a-4f7d-9afa-cdd5b5fece45 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d9922026c1784ed4bd40bbb5c139cf04)` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:31:45.048179 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740412b5-e09a-4f7d-9afa-cdd5b5fece45 admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:31:45.245629 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:31:45.252374 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-740412b5-e09a-4f7d-9afa-cdd5b5fece45 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd9922026c1784ed4bd40bbb5c139cf04'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:31:45.267899 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-740412b5-e09a-4f7d-9afa-cdd5b5fece45 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd9922026c1784ed4bd40bbb5c139cf04'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:31:45.268767 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-740412b5-e09a-4f7d-9afa-cdd5b5fece45 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd9922026c1784ed4bd40bbb5c139cf04'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:31:45.349897 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-740412b5-e09a-4f7d-9afa-cdd5b5fece45 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd9922026c1784ed4bd40bbb5c139cf04'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:31:45.358977 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-740412b5-e09a-4f7d-9afa-cdd5b5fece45 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd9922026c1784ed4bd40bbb5c139cf04'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:31:45.366944 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1426/2834] 10.222.0.23 () {58 vars in 1223 bytes} [Thu May 19 12:31:44 2022] DELETE /identity/v3/users/d9922026c1784ed4bd40bbb5c139cf04 => generated 0 bytes in 412 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:31:45.373278 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4905fa1-05df-4aaa-8ec6-fb64cbb426b7 None None] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:31:45.373876 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4905fa1-05df-4aaa-8ec6-fb64cbb426b7 None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:31:45.373876 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4905fa1-05df-4aaa-8ec6-fb64cbb426b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:31:45.392604 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.auth.core [None req-f4905fa1-05df-4aaa-8ec6-fb64cbb426b7 None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:31:45.393662 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-f4905fa1-05df-4aaa-8ec6-fb64cbb426b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:31:45.476901 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1409/2835] 10.222.0.23 () {58 vars in 957 bytes} [Thu May 19 12:31:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:31:45.489955 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740412b5-e09a-4f7d-9afa-cdd5b5fece45 admin admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:31:45.492489 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-740412b5-e09a-4f7d-9afa-cdd5b5fece45 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:31:45.569541 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740412b5-e09a-4f7d-9afa-cdd5b5fece45 admin admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:31:45.574621 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8382b9-c48d-4b1b-84ea-4ee6581e40c8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a00b3fa7497456895e5e0bf105c4fc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd70d25bf69314820b8aaecb65576978a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:31:45.575301 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8382b9-c48d-4b1b-84ea-4ee6581e40c8 service neutron] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:31:45.575585 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8382b9-c48d-4b1b-84ea-4ee6581e40c8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:31:45.575904 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8382b9-c48d-4b1b-84ea-4ee6581e40c8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:31:45.587101 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8382b9-c48d-4b1b-84ea-4ee6581e40c8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:31:45.589294 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8382b9-c48d-4b1b-84ea-4ee6581e40c8 service neutron] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:31:45.600006 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1427/2836] 10.222.0.23 () {60 vars in 1365 bytes} [Thu May 19 12:31:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:31:45.921005 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4905fa1-05df-4aaa-8ec6-fb64cbb426b7 None None] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:31:45.923580 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-f4905fa1-05df-4aaa-8ec6-fb64cbb426b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:31:45.946184 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4905fa1-05df-4aaa-8ec6-fb64cbb426b7 None None] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:31:45.953212 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207315c7-3cee-44d7-beed-a1a0196e109c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:31:45.954120 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207315c7-3cee-44d7-beed-a1a0196e109c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:31:45.954431 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207315c7-3cee-44d7-beed-a1a0196e109c admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:31:45.954728 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207315c7-3cee-44d7-beed-a1a0196e109c admin admin] PATH_INFO: `/v3/projects/03c8ec22ac914f63a81e67c98d018815` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:31:45.956051 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207315c7-3cee-44d7-beed-a1a0196e109c admin admin] RBAC: Authorizing `identity:delete_project(project_id=03c8ec22ac914f63a81e67c98d018815)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:31:45.958074 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207315c7-3cee-44d7-beed-a1a0196e109c admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:31:46.115481 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-207315c7-3cee-44d7-beed-a1a0196e109c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '03c8ec22ac914f63a81e67c98d018815', 'request_id': 'req-207315c7-3cee-44d7-beed-a1a0196e109c'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:31:46.122777 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-207315c7-3cee-44d7-beed-a1a0196e109c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '03c8ec22ac914f63a81e67c98d018815', 'request_id': 'req-207315c7-3cee-44d7-beed-a1a0196e109c'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:31:46.219701 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-207315c7-3cee-44d7-beed-a1a0196e109c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '03c8ec22ac914f63a81e67c98d018815', 'request_id': 'req-207315c7-3cee-44d7-beed-a1a0196e109c'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:31:46.233614 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-207315c7-3cee-44d7-beed-a1a0196e109c admin admin] The token cache is being invalidate because project 03c8ec22ac914f63a81e67c98d018815 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80539) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 12:31:46.233908 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-207315c7-3cee-44d7-beed-a1a0196e109c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:31:46.234691 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-207315c7-3cee-44d7-beed-a1a0196e109c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:31:46.236480 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1410/2837] 10.222.0.23 () {58 vars in 1232 bytes} [Thu May 19 12:31:45 2022] DELETE /identity/v3/projects/03c8ec22ac914f63a81e67c98d018815 => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:31:54.323610 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8382b9-c48d-4b1b-84ea-4ee6581e40c8 service neutron] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:31:54.325405 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-3a8382b9-c48d-4b1b-84ea-4ee6581e40c8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:31:54.451337 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8382b9-c48d-4b1b-84ea-4ee6581e40c8 service neutron] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:31:54.506839 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:DELETE=5,keystone:SELECT=94,keystone:INSERT=1 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:31:54.535275 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510b9cc4-bfb6-4e60-97af-feac9b35543b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50f937eed4f3414fa6f19cab2a84eb5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd70d25bf69314820b8aaecb65576978a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:31:54.536202 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510b9cc4-bfb6-4e60-97af-feac9b35543b service placement] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:31:54.536592 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510b9cc4-bfb6-4e60-97af-feac9b35543b service placement] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:31:54.536982 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510b9cc4-bfb6-4e60-97af-feac9b35543b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:31:54.539257 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-510b9cc4-bfb6-4e60-97af-feac9b35543b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:31:54.555326 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510b9cc4-bfb6-4e60-97af-feac9b35543b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:31:54.559862 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510b9cc4-bfb6-4e60-97af-feac9b35543b service placement] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:31:54.579577 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1428/2838] 10.222.0.23 () {60 vars in 1365 bytes} [Thu May 19 12:31:54 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 256 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:31:56.230221 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=54,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:31:57.818706 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207315c7-3cee-44d7-beed-a1a0196e109c admin admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:31:57.820505 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-207315c7-3cee-44d7-beed-a1a0196e109c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:31:57.914948 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207315c7-3cee-44d7-beed-a1a0196e109c admin admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:31:57.920438 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da16617d-0f0b-4dd0-bc9c-a6523100ae13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:31:57.921339 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da16617d-0f0b-4dd0-bc9c-a6523100ae13 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:31:57.921642 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da16617d-0f0b-4dd0-bc9c-a6523100ae13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:31:57.922035 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da16617d-0f0b-4dd0-bc9c-a6523100ae13 admin admin] PATH_INFO: `/v3/users/5a22b181786d47369715c98416f7bae8` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:31:57.923328 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da16617d-0f0b-4dd0-bc9c-a6523100ae13 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5a22b181786d47369715c98416f7bae8)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:31:57.925346 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da16617d-0f0b-4dd0-bc9c-a6523100ae13 admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:31:58.295224 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-da16617d-0f0b-4dd0-bc9c-a6523100ae13 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5a22b181786d47369715c98416f7bae8'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:31:58.312377 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-da16617d-0f0b-4dd0-bc9c-a6523100ae13 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5a22b181786d47369715c98416f7bae8'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:31:58.313224 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-da16617d-0f0b-4dd0-bc9c-a6523100ae13 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5a22b181786d47369715c98416f7bae8'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:31:58.573977 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-da16617d-0f0b-4dd0-bc9c-a6523100ae13 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5a22b181786d47369715c98416f7bae8'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:31:58.582436 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-da16617d-0f0b-4dd0-bc9c-a6523100ae13 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5a22b181786d47369715c98416f7bae8'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:31:58.589905 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1411/2839] 10.222.0.23 () {58 vars in 1223 bytes} [Thu May 19 12:31:57 2022] DELETE /identity/v3/users/5a22b181786d47369715c98416f7bae8 => generated 0 bytes in 772 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:31:58.594682 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9963f2-7aa2-4c02-a993-1af56bd621e5 None None] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:31:58.595266 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9963f2-7aa2-4c02-a993-1af56bd621e5 None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:31:58.595266 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9963f2-7aa2-4c02-a993-1af56bd621e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:31:58.615077 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.auth.core [None req-2a9963f2-7aa2-4c02-a993-1af56bd621e5 None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:31:58.623278 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-2a9963f2-7aa2-4c02-a993-1af56bd621e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:31:58.682817 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1429/2840] 10.222.0.23 () {58 vars in 957 bytes} [Thu May 19 12:31:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:31:58.693944 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da16617d-0f0b-4dd0-bc9c-a6523100ae13 admin admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:31:58.695779 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-da16617d-0f0b-4dd0-bc9c-a6523100ae13 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:31:58.776924 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da16617d-0f0b-4dd0-bc9c-a6523100ae13 admin admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:31:58.784558 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4533ccc3-7e3b-48bc-83b5-65cd91907164 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a00b3fa7497456895e5e0bf105c4fc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd70d25bf69314820b8aaecb65576978a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:31:58.785237 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4533ccc3-7e3b-48bc-83b5-65cd91907164 service neutron] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:31:58.785524 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4533ccc3-7e3b-48bc-83b5-65cd91907164 service neutron] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:31:58.785805 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4533ccc3-7e3b-48bc-83b5-65cd91907164 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:31:58.797284 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4533ccc3-7e3b-48bc-83b5-65cd91907164 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:31:58.798755 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4533ccc3-7e3b-48bc-83b5-65cd91907164 service neutron] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:31:58.809823 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1412/2841] 10.222.0.23 () {60 vars in 1365 bytes} [Thu May 19 12:31:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:31:59.737100 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9963f2-7aa2-4c02-a993-1af56bd621e5 None None] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:31:59.739007 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-2a9963f2-7aa2-4c02-a993-1af56bd621e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:31:59.757577 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9963f2-7aa2-4c02-a993-1af56bd621e5 None None] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:31:59.765685 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124a474b-55ea-4652-825e-3c73fbbb48d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:31:59.794233 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124a474b-55ea-4652-825e-3c73fbbb48d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:31:59.794233 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124a474b-55ea-4652-825e-3c73fbbb48d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:31:59.794233 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124a474b-55ea-4652-825e-3c73fbbb48d6 admin admin] PATH_INFO: `/v3/projects/c939c19350bf409f9e11612020da5b9f` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:31:59.794233 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124a474b-55ea-4652-825e-3c73fbbb48d6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c939c19350bf409f9e11612020da5b9f)` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:31:59.794233 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124a474b-55ea-4652-825e-3c73fbbb48d6 admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:32:00.024793 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-124a474b-55ea-4652-825e-3c73fbbb48d6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c939c19350bf409f9e11612020da5b9f', 'request_id': 'req-124a474b-55ea-4652-825e-3c73fbbb48d6'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:32:00.034771 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-124a474b-55ea-4652-825e-3c73fbbb48d6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c939c19350bf409f9e11612020da5b9f', 'request_id': 'req-124a474b-55ea-4652-825e-3c73fbbb48d6'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:32:00.045101 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-124a474b-55ea-4652-825e-3c73fbbb48d6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c939c19350bf409f9e11612020da5b9f', 'request_id': 'req-124a474b-55ea-4652-825e-3c73fbbb48d6'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:32:00.201149 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-124a474b-55ea-4652-825e-3c73fbbb48d6 admin admin] The token cache is being invalidate because project c939c19350bf409f9e11612020da5b9f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80540) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 12:32:00.201489 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-124a474b-55ea-4652-825e-3c73fbbb48d6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:32:00.202239 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-124a474b-55ea-4652-825e-3c73fbbb48d6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:32:00.204038 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1430/2842] 10.222.0.23 () {58 vars in 1232 bytes} [Thu May 19 12:31:59 2022] DELETE /identity/v3/projects/c939c19350bf409f9e11612020da5b9f => generated 0 bytes in 468 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:32:08.804826 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=79,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:32:10.197657 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=67,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:32:21.528093 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4533ccc3-7e3b-48bc-83b5-65cd91907164 service neutron] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:32:21.529537 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-4533ccc3-7e3b-48bc-83b5-65cd91907164 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:32:21.542183 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4533ccc3-7e3b-48bc-83b5-65cd91907164 service neutron] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:32:21.597768 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f110e2-497e-4dc4-9efc-9e9040ac9087 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:32:21.598645 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f110e2-497e-4dc4-9efc-9e9040ac9087 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:32:21.598925 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f110e2-497e-4dc4-9efc-9e9040ac9087 admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:32:21.599200 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f110e2-497e-4dc4-9efc-9e9040ac9087 admin admin] PATH_INFO: `/v3/users/5508b241f22d4208a8efe645509266ac` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:32:21.600426 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f110e2-497e-4dc4-9efc-9e9040ac9087 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5508b241f22d4208a8efe645509266ac)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:32:21.602395 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f110e2-497e-4dc4-9efc-9e9040ac9087 admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:32:21.808904 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-33f110e2-497e-4dc4-9efc-9e9040ac9087 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5508b241f22d4208a8efe645509266ac'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:32:21.821652 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-33f110e2-497e-4dc4-9efc-9e9040ac9087 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5508b241f22d4208a8efe645509266ac'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:32:21.822369 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-33f110e2-497e-4dc4-9efc-9e9040ac9087 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5508b241f22d4208a8efe645509266ac'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:32:21.937568 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-33f110e2-497e-4dc4-9efc-9e9040ac9087 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5508b241f22d4208a8efe645509266ac'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:32:21.946840 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-33f110e2-497e-4dc4-9efc-9e9040ac9087 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5508b241f22d4208a8efe645509266ac'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:32:21.956047 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1413/2843] 10.222.0.23 () {58 vars in 1223 bytes} [Thu May 19 12:32:21 2022] DELETE /identity/v3/users/5508b241f22d4208a8efe645509266ac => generated 0 bytes in 429 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 12:32:21.963168 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337665c9-bc1f-4841-8a7f-5d1b88927efa None None] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:32:21.963740 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337665c9-bc1f-4841-8a7f-5d1b88927efa None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:32:21.963740 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337665c9-bc1f-4841-8a7f-5d1b88927efa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:32:21.983440 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.auth.core [None req-337665c9-bc1f-4841-8a7f-5d1b88927efa None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:32:21.984790 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-337665c9-bc1f-4841-8a7f-5d1b88927efa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:32:22.067223 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1431/2844] 10.222.0.23 () {58 vars in 957 bytes} [Thu May 19 12:32:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:32:22.077591 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f110e2-497e-4dc4-9efc-9e9040ac9087 admin admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:32:22.079073 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-33f110e2-497e-4dc4-9efc-9e9040ac9087 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:32:22.141636 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f110e2-497e-4dc4-9efc-9e9040ac9087 admin admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:32:22.145587 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb09164b-bff4-4aba-ab60-02fc48b5c64b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a00b3fa7497456895e5e0bf105c4fc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd70d25bf69314820b8aaecb65576978a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:32:22.146258 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb09164b-bff4-4aba-ab60-02fc48b5c64b service neutron] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:32:22.146549 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb09164b-bff4-4aba-ab60-02fc48b5c64b service neutron] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:32:22.146828 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb09164b-bff4-4aba-ab60-02fc48b5c64b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:32:22.157499 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb09164b-bff4-4aba-ab60-02fc48b5c64b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:32:22.158936 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb09164b-bff4-4aba-ab60-02fc48b5c64b service neutron] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:32:22.168716 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1414/2845] 10.222.0.23 () {60 vars in 1365 bytes} [Thu May 19 12:32:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 12:32:22.621895 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337665c9-bc1f-4841-8a7f-5d1b88927efa None None] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:32:22.625536 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-337665c9-bc1f-4841-8a7f-5d1b88927efa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:32:22.648514 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337665c9-bc1f-4841-8a7f-5d1b88927efa None None] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:32:22.658132 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fec96bb-ea06-4902-8c4b-54ba37e0e625 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:32:22.659711 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fec96bb-ea06-4902-8c4b-54ba37e0e625 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:32:22.660261 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fec96bb-ea06-4902-8c4b-54ba37e0e625 admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:32:22.660768 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fec96bb-ea06-4902-8c4b-54ba37e0e625 admin admin] PATH_INFO: `/v3/projects/753c6657bdfb409dab2c4bdd006818b8` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:32:22.662968 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fec96bb-ea06-4902-8c4b-54ba37e0e625 admin admin] RBAC: Authorizing `identity:delete_project(project_id=753c6657bdfb409dab2c4bdd006818b8)` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:32:22.666458 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fec96bb-ea06-4902-8c4b-54ba37e0e625 admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:32:22.770179 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-3fec96bb-ea06-4902-8c4b-54ba37e0e625 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '753c6657bdfb409dab2c4bdd006818b8', 'request_id': 'req-3fec96bb-ea06-4902-8c4b-54ba37e0e625'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:32:22.777480 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-3fec96bb-ea06-4902-8c4b-54ba37e0e625 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '753c6657bdfb409dab2c4bdd006818b8', 'request_id': 'req-3fec96bb-ea06-4902-8c4b-54ba37e0e625'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:32:22.783112 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-3fec96bb-ea06-4902-8c4b-54ba37e0e625 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '753c6657bdfb409dab2c4bdd006818b8', 'request_id': 'req-3fec96bb-ea06-4902-8c4b-54ba37e0e625'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:32:22.859656 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-3fec96bb-ea06-4902-8c4b-54ba37e0e625 admin admin] The token cache is being invalidate because project 753c6657bdfb409dab2c4bdd006818b8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80540) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 12:32:22.859978 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-3fec96bb-ea06-4902-8c4b-54ba37e0e625 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:32:22.860617 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-3fec96bb-ea06-4902-8c4b-54ba37e0e625 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:32:22.862194 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1432/2846] 10.222.0.23 () {58 vars in 1232 bytes} [Thu May 19 12:32:22 2022] DELETE /identity/v3/projects/753c6657bdfb409dab2c4bdd006818b8 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:32:22.867590 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce04927-d995-4453-9186-08f45226af0a None None] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:32:22.868855 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce04927-d995-4453-9186-08f45226af0a None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:32:22.868855 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce04927-d995-4453-9186-08f45226af0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:32:22.888259 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.auth.core [None req-5ce04927-d995-4453-9186-08f45226af0a None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:32:22.889507 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-5ce04927-d995-4453-9186-08f45226af0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:32:22.907914 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:32:22.954053 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1415/2847] 10.222.0.23 () {58 vars in 957 bytes} [Thu May 19 12:32:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:32:22.960873 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5e226e-c84e-4b87-9c6b-ac76f7a03bcc None None] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:32:22.961188 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5e226e-c84e-4b87-9c6b-ac76f7a03bcc None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:32:22.962199 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5e226e-c84e-4b87-9c6b-ac76f7a03bcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:32:22.982925 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.auth.core [None req-bc5e226e-c84e-4b87-9c6b-ac76f7a03bcc None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:32:22.984183 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-bc5e226e-c84e-4b87-9c6b-ac76f7a03bcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:32:22.992465 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1433/2848] 10.222.0.23 () {58 vars in 957 bytes} [Thu May 19 12:32:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:32:22.996777 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce04927-d995-4453-9186-08f45226af0a None None] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:32:23.006497 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce04927-d995-4453-9186-08f45226af0a None None] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:32:23.012198 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c9b8d0-1481-430f-b45f-c79c591ea139 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:32:23.016969 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c9b8d0-1481-430f-b45f-c79c591ea139 admin admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:32:23.016969 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c9b8d0-1481-430f-b45f-c79c591ea139 admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:32:23.016969 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c9b8d0-1481-430f-b45f-c79c591ea139 admin admin] PATH_INFO: `/v3/domains` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:32:23.016969 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c9b8d0-1481-430f-b45f-c79c591ea139 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:32:23.016969 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c9b8d0-1481-430f-b45f-c79c591ea139 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:32:23.016969 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c9b8d0-1481-430f-b45f-c79c591ea139 admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:32:23.029189 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1416/2849] 10.222.0.23 () {58 vars in 1152 bytes} [Thu May 19 12:32:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:32:23.033599 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5e226e-c84e-4b87-9c6b-ac76f7a03bcc None None] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:32:23.043261 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5e226e-c84e-4b87-9c6b-ac76f7a03bcc None None] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:32:23.047865 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b73a60-9728-4a12-8cb8-7095b641aa0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:32:23.048517 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b73a60-9728-4a12-8cb8-7095b641aa0b admin admin] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:32:23.048797 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b73a60-9728-4a12-8cb8-7095b641aa0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:32:23.049071 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b73a60-9728-4a12-8cb8-7095b641aa0b admin admin] PATH_INFO: `/v3/projects` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:32:23.049758 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b73a60-9728-4a12-8cb8-7095b641aa0b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:32:23.051642 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b73a60-9728-4a12-8cb8-7095b641aa0b admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:32:23.158924 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1434/2850] 10.222.0.23 () {60 vars in 1152 bytes} [Thu May 19 12:32:23 2022] POST /identity/v3/projects => generated 373 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 12:32:23.163048 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c9b8d0-1481-430f-b45f-c79c591ea139 admin admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:32:23.173737 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c9b8d0-1481-430f-b45f-c79c591ea139 admin admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:32:23.231948 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac203ea-981d-4cfc-9e8d-99ebb7d2f3b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:32:23.232641 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac203ea-981d-4cfc-9e8d-99ebb7d2f3b4 admin admin] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:32:23.232930 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac203ea-981d-4cfc-9e8d-99ebb7d2f3b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:32:23.233210 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac203ea-981d-4cfc-9e8d-99ebb7d2f3b4 admin admin] PATH_INFO: `/v3/users` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:32:23.233919 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac203ea-981d-4cfc-9e8d-99ebb7d2f3b4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:32:23.236031 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac203ea-981d-4cfc-9e8d-99ebb7d2f3b4 admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:32:23.321581 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1417/2851] 10.222.0.23 () {60 vars in 1143 bytes} [Thu May 19 12:32:23 2022] POST /identity/v3/users => generated 287 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 12:32:23.326163 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b73a60-9728-4a12-8cb8-7095b641aa0b admin admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:32:23.336958 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b73a60-9728-4a12-8cb8-7095b641aa0b admin admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:32:23.342229 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca30464-81d7-4241-8f6b-7daa79339967 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:32:23.342911 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca30464-81d7-4241-8f6b-7daa79339967 admin admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:32:23.343200 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca30464-81d7-4241-8f6b-7daa79339967 admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:32:23.343487 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca30464-81d7-4241-8f6b-7daa79339967 admin admin] PATH_INFO: `/v3/roles` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:32:23.344268 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca30464-81d7-4241-8f6b-7daa79339967 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:32:23.345902 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca30464-81d7-4241-8f6b-7daa79339967 admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:32:23.358568 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1435/2852] 10.222.0.23 () {58 vars in 1121 bytes} [Thu May 19 12:32:23 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 12:32:23.362608 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac203ea-981d-4cfc-9e8d-99ebb7d2f3b4 admin admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:32:23.374181 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac203ea-981d-4cfc-9e8d-99ebb7d2f3b4 admin admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:32:23.379200 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc4eba0-a3bb-4518-aa1b-d2cbabc45533 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:32:23.379936 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc4eba0-a3bb-4518-aa1b-d2cbabc45533 admin admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:32:23.380223 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc4eba0-a3bb-4518-aa1b-d2cbabc45533 admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:32:23.380501 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc4eba0-a3bb-4518-aa1b-d2cbabc45533 admin admin] PATH_INFO: `/v3/roles` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:32:23.381232 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc4eba0-a3bb-4518-aa1b-d2cbabc45533 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:32:23.382893 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc4eba0-a3bb-4518-aa1b-d2cbabc45533 admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:32:23.395235 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1418/2853] 10.222.0.23 () {58 vars in 1121 bytes} [Thu May 19 12:32:23 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 12:32:23.399273 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca30464-81d7-4241-8f6b-7daa79339967 admin admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:32:23.410013 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca30464-81d7-4241-8f6b-7daa79339967 admin admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:32:23.414604 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404dc497-fc5e-4910-81a1-67127b664d02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:32:23.415688 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404dc497-fc5e-4910-81a1-67127b664d02 admin admin] REQUEST_METHOD: `PUT` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:32:23.416006 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404dc497-fc5e-4910-81a1-67127b664d02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:32:23.416280 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404dc497-fc5e-4910-81a1-67127b664d02 admin admin] PATH_INFO: `/v3/projects/c8ba7f082cf24c4190b30294241e5130/users/beaf253a429345e08141e28a003d12b1/roles/259eb400cdb4439db72d14456ae9cd08` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:32:23.432730 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404dc497-fc5e-4910-81a1-67127b664d02 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c8ba7f082cf24c4190b30294241e5130, user_id=beaf253a429345e08141e28a003d12b1, role_id=259eb400cdb4439db72d14456ae9cd08)` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:32:23.435434 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404dc497-fc5e-4910-81a1-67127b664d02 admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:32:23.520769 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1436/2854] 10.222.0.23 () {60 vars in 1482 bytes} [Thu May 19 12:32:23 2022] PUT /identity/v3/projects/c8ba7f082cf24c4190b30294241e5130/users/beaf253a429345e08141e28a003d12b1/roles/259eb400cdb4439db72d14456ae9cd08 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:32:23.532269 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63dd555-9e54-43f8-b2a9-8de20ae1d2a6 None None] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:32:23.532566 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63dd555-9e54-43f8-b2a9-8de20ae1d2a6 None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:32:23.532883 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63dd555-9e54-43f8-b2a9-8de20ae1d2a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:32:23.562477 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.auth.core [None req-a63dd555-9e54-43f8-b2a9-8de20ae1d2a6 None None] MFA Rules not processed for user `beaf253a429345e08141e28a003d12b1`. Rule list: `[]` (Enabled: `True`). {{(pid=80539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:32:23.563741 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-a63dd555-9e54-43f8-b2a9-8de20ae1d2a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:32:23.632746 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1419/2855] 10.222.0.23 () {58 vars in 957 bytes} [Thu May 19 12:32:23 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:32:23.651656 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404dc497-fc5e-4910-81a1-67127b664d02 admin admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:32:23.653520 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-404dc497-fc5e-4910-81a1-67127b664d02 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:32:23.677098 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:32:23.714496 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404dc497-fc5e-4910-81a1-67127b664d02 admin admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:32:23.718233 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd56ff73-40d8-4fc1-a83d-773c297bcffd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8800254c0e784e42a791fb9abc274d79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd70d25bf69314820b8aaecb65576978a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:32:23.719009 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd56ff73-40d8-4fc1-a83d-773c297bcffd service cinder] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:32:23.719524 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd56ff73-40d8-4fc1-a83d-773c297bcffd service cinder] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:32:23.720073 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd56ff73-40d8-4fc1-a83d-773c297bcffd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:32:23.730219 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd56ff73-40d8-4fc1-a83d-773c297bcffd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:32:23.731674 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd56ff73-40d8-4fc1-a83d-773c297bcffd service cinder] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:32:23.741316 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1437/2856] 10.222.0.23 () {60 vars in 1365 bytes} [Thu May 19 12:32:23 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:32:23.767406 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63dd555-9e54-43f8-b2a9-8de20ae1d2a6 None None] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:32:23.789647 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:32:23.838126 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63dd555-9e54-43f8-b2a9-8de20ae1d2a6 None None] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:32:23.842986 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582abe6a-4a85-4f89-b720-5810894dea97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:32:23.843888 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582abe6a-4a85-4f89-b720-5810894dea97 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:32:23.844208 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582abe6a-4a85-4f89-b720-5810894dea97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:32:23.844449 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582abe6a-4a85-4f89-b720-5810894dea97 admin admin] PATH_INFO: `/v3/users/beaf253a429345e08141e28a003d12b1` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:32:23.845680 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582abe6a-4a85-4f89-b720-5810894dea97 admin admin] RBAC: Authorizing `identity:delete_user(user_id=beaf253a429345e08141e28a003d12b1)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:32:23.847630 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582abe6a-4a85-4f89-b720-5810894dea97 admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:32:24.078718 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-582abe6a-4a85-4f89-b720-5810894dea97 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'beaf253a429345e08141e28a003d12b1'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:32:24.098318 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-582abe6a-4a85-4f89-b720-5810894dea97 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'beaf253a429345e08141e28a003d12b1'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:32:24.102008 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-582abe6a-4a85-4f89-b720-5810894dea97 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'beaf253a429345e08141e28a003d12b1'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:32:24.185057 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-582abe6a-4a85-4f89-b720-5810894dea97 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'beaf253a429345e08141e28a003d12b1'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:32:24.193743 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-582abe6a-4a85-4f89-b720-5810894dea97 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'beaf253a429345e08141e28a003d12b1'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:32:24.200782 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1420/2857] 10.222.0.23 () {58 vars in 1223 bytes} [Thu May 19 12:32:23 2022] DELETE /identity/v3/users/beaf253a429345e08141e28a003d12b1 => generated 0 bytes in 434 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:32:24.205340 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34df3afc-dc89-405a-85dd-54cd905e4d05 None None] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:32:24.205871 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34df3afc-dc89-405a-85dd-54cd905e4d05 None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:32:24.206093 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34df3afc-dc89-405a-85dd-54cd905e4d05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:32:24.225559 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.auth.core [None req-34df3afc-dc89-405a-85dd-54cd905e4d05 None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:32:24.226818 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-34df3afc-dc89-405a-85dd-54cd905e4d05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:32:24.289399 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1438/2858] 10.222.0.23 () {58 vars in 957 bytes} [Thu May 19 12:32:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:32:24.299615 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582abe6a-4a85-4f89-b720-5810894dea97 admin admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:32:24.301223 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-582abe6a-4a85-4f89-b720-5810894dea97 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:32:24.369144 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582abe6a-4a85-4f89-b720-5810894dea97 admin admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:32:24.373553 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96b5320-ced8-4ce4-8494-6519c5c143e3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a00b3fa7497456895e5e0bf105c4fc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd70d25bf69314820b8aaecb65576978a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:32:24.374207 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96b5320-ced8-4ce4-8494-6519c5c143e3 service neutron] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:32:24.374490 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96b5320-ced8-4ce4-8494-6519c5c143e3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:32:24.374765 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96b5320-ced8-4ce4-8494-6519c5c143e3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:32:24.386342 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96b5320-ced8-4ce4-8494-6519c5c143e3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:32:24.387841 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96b5320-ced8-4ce4-8494-6519c5c143e3 service neutron] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:32:24.398399 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1421/2859] 10.222.0.23 () {60 vars in 1365 bytes} [Thu May 19 12:32:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:32:24.837148 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34df3afc-dc89-405a-85dd-54cd905e4d05 None None] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:32:24.839371 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-34df3afc-dc89-405a-85dd-54cd905e4d05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:32:24.855512 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34df3afc-dc89-405a-85dd-54cd905e4d05 None None] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:32:24.862378 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7cd0d6-0e54-4cea-b3bd-6e291f866626 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:32:24.864184 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7cd0d6-0e54-4cea-b3bd-6e291f866626 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:32:24.864685 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7cd0d6-0e54-4cea-b3bd-6e291f866626 admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:32:24.865170 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7cd0d6-0e54-4cea-b3bd-6e291f866626 admin admin] PATH_INFO: `/v3/projects/c8ba7f082cf24c4190b30294241e5130` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:32:24.867470 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7cd0d6-0e54-4cea-b3bd-6e291f866626 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c8ba7f082cf24c4190b30294241e5130)` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:32:24.872817 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7cd0d6-0e54-4cea-b3bd-6e291f866626 admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:32:25.045528 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-1b7cd0d6-0e54-4cea-b3bd-6e291f866626 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c8ba7f082cf24c4190b30294241e5130', 'request_id': 'req-1b7cd0d6-0e54-4cea-b3bd-6e291f866626'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:32:25.055229 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-1b7cd0d6-0e54-4cea-b3bd-6e291f866626 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c8ba7f082cf24c4190b30294241e5130', 'request_id': 'req-1b7cd0d6-0e54-4cea-b3bd-6e291f866626'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:32:25.067274 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-1b7cd0d6-0e54-4cea-b3bd-6e291f866626 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c8ba7f082cf24c4190b30294241e5130', 'request_id': 'req-1b7cd0d6-0e54-4cea-b3bd-6e291f866626'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:32:25.180344 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-1b7cd0d6-0e54-4cea-b3bd-6e291f866626 admin admin] The token cache is being invalidate because project c8ba7f082cf24c4190b30294241e5130 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80540) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 12:32:25.180739 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-1b7cd0d6-0e54-4cea-b3bd-6e291f866626 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:32:25.181228 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-1b7cd0d6-0e54-4cea-b3bd-6e291f866626 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:32:25.183494 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1439/2860] 10.222.0.23 () {58 vars in 1232 bytes} [Thu May 19 12:32:24 2022] DELETE /identity/v3/projects/c8ba7f082cf24c4190b30294241e5130 => generated 0 bytes in 347 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:32:25.188688 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027858fb-b450-497e-9339-981e84da4b0f None None] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:32:25.193441 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027858fb-b450-497e-9339-981e84da4b0f None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:32:25.193441 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027858fb-b450-497e-9339-981e84da4b0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:32:25.208185 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.auth.core [None req-027858fb-b450-497e-9339-981e84da4b0f None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:32:25.209428 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-027858fb-b450-497e-9339-981e84da4b0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:32:25.258545 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:32:25.294020 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1422/2861] 10.222.0.23 () {58 vars in 957 bytes} [Thu May 19 12:32:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:32:25.301934 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fad6d84-b0c8-4f52-b940-057e57aa05a1 None None] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:32:25.302784 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fad6d84-b0c8-4f52-b940-057e57aa05a1 None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:32:25.302784 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fad6d84-b0c8-4f52-b940-057e57aa05a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:32:25.322959 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.auth.core [None req-2fad6d84-b0c8-4f52-b940-057e57aa05a1 None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:32:25.324291 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-2fad6d84-b0c8-4f52-b940-057e57aa05a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:32:25.332522 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1440/2862] 10.222.0.23 () {58 vars in 957 bytes} [Thu May 19 12:32:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:32:25.338663 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027858fb-b450-497e-9339-981e84da4b0f None None] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:32:25.349554 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027858fb-b450-497e-9339-981e84da4b0f None None] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:32:25.355816 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762be3be-de8a-48ad-974d-49c2d7010229 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:32:25.356645 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762be3be-de8a-48ad-974d-49c2d7010229 admin admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:32:25.357066 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762be3be-de8a-48ad-974d-49c2d7010229 admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:32:25.357396 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762be3be-de8a-48ad-974d-49c2d7010229 admin admin] PATH_INFO: `/v3/domains` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:32:25.358211 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762be3be-de8a-48ad-974d-49c2d7010229 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:32:25.358564 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762be3be-de8a-48ad-974d-49c2d7010229 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:32:25.360419 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762be3be-de8a-48ad-974d-49c2d7010229 admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:32:25.375746 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1423/2863] 10.222.0.23 () {58 vars in 1152 bytes} [Thu May 19 12:32:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:32:25.380777 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fad6d84-b0c8-4f52-b940-057e57aa05a1 None None] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:32:25.390936 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fad6d84-b0c8-4f52-b940-057e57aa05a1 None None] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:32:25.396095 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7632bae0-ac8a-49aa-b2d4-3168d6ea155a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:32:25.396967 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7632bae0-ac8a-49aa-b2d4-3168d6ea155a admin admin] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:32:25.397368 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7632bae0-ac8a-49aa-b2d4-3168d6ea155a admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:32:25.397697 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7632bae0-ac8a-49aa-b2d4-3168d6ea155a admin admin] PATH_INFO: `/v3/projects` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:32:25.398450 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7632bae0-ac8a-49aa-b2d4-3168d6ea155a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:32:25.400587 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7632bae0-ac8a-49aa-b2d4-3168d6ea155a admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:32:25.531250 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1441/2864] 10.222.0.23 () {60 vars in 1152 bytes} [Thu May 19 12:32:25 2022] POST /identity/v3/projects => generated 383 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 12:32:25.536277 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762be3be-de8a-48ad-974d-49c2d7010229 admin admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:32:25.547316 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762be3be-de8a-48ad-974d-49c2d7010229 admin admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:32:25.622283 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee56e52f-55a7-42c4-b5ad-866b9762be00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:32:25.623497 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee56e52f-55a7-42c4-b5ad-866b9762be00 admin admin] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:32:25.624036 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee56e52f-55a7-42c4-b5ad-866b9762be00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:32:25.624527 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee56e52f-55a7-42c4-b5ad-866b9762be00 admin admin] PATH_INFO: `/v3/users` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:32:25.625836 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee56e52f-55a7-42c4-b5ad-866b9762be00 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:32:25.629650 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee56e52f-55a7-42c4-b5ad-866b9762be00 admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:32:25.740574 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1424/2865] 10.222.0.23 () {60 vars in 1143 bytes} [Thu May 19 12:32:25 2022] POST /identity/v3/users => generated 292 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 12:32:25.747577 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7632bae0-ac8a-49aa-b2d4-3168d6ea155a admin admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:32:25.758850 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7632bae0-ac8a-49aa-b2d4-3168d6ea155a admin admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:32:25.764556 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ade5730-0e46-4d3e-8e2a-dd98d89ecc07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:32:25.765319 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ade5730-0e46-4d3e-8e2a-dd98d89ecc07 admin admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:32:25.765679 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ade5730-0e46-4d3e-8e2a-dd98d89ecc07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:32:25.766005 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ade5730-0e46-4d3e-8e2a-dd98d89ecc07 admin admin] PATH_INFO: `/v3/roles` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:32:25.766808 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ade5730-0e46-4d3e-8e2a-dd98d89ecc07 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:32:25.768620 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ade5730-0e46-4d3e-8e2a-dd98d89ecc07 admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:32:25.780988 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1442/2866] 10.222.0.23 () {58 vars in 1121 bytes} [Thu May 19 12:32:25 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 12:32:25.785616 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee56e52f-55a7-42c4-b5ad-866b9762be00 admin admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:32:25.796596 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee56e52f-55a7-42c4-b5ad-866b9762be00 admin admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:32:25.801576 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3eda3b0-e13c-49c4-9eb3-45eaa322ebc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:32:25.802263 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3eda3b0-e13c-49c4-9eb3-45eaa322ebc8 admin admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:32:25.802551 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3eda3b0-e13c-49c4-9eb3-45eaa322ebc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:32:25.802834 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3eda3b0-e13c-49c4-9eb3-45eaa322ebc8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:32:25.803584 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3eda3b0-e13c-49c4-9eb3-45eaa322ebc8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:32:25.805441 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3eda3b0-e13c-49c4-9eb3-45eaa322ebc8 admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:32:25.817832 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1425/2867] 10.222.0.23 () {58 vars in 1121 bytes} [Thu May 19 12:32:25 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 19 12:32:25.822424 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ade5730-0e46-4d3e-8e2a-dd98d89ecc07 admin admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:32:25.833753 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ade5730-0e46-4d3e-8e2a-dd98d89ecc07 admin admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:32:25.839729 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e056359-dbe5-4355-bcb8-866bf0ce7281 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:32:25.840139 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e056359-dbe5-4355-bcb8-866bf0ce7281 admin admin] REQUEST_METHOD: `PUT` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:32:25.840424 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e056359-dbe5-4355-bcb8-866bf0ce7281 admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:32:25.840705 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e056359-dbe5-4355-bcb8-866bf0ce7281 admin admin] PATH_INFO: `/v3/projects/b0b15da1cdfb45c88bb7827e650d33c9/users/f901fc3ba9b24c39a252aa5a7c9d431c/roles/259eb400cdb4439db72d14456ae9cd08` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:32:25.843406 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:32:25.857830 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e056359-dbe5-4355-bcb8-866bf0ce7281 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b0b15da1cdfb45c88bb7827e650d33c9, user_id=f901fc3ba9b24c39a252aa5a7c9d431c, role_id=259eb400cdb4439db72d14456ae9cd08)` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:32:25.862674 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e056359-dbe5-4355-bcb8-866bf0ce7281 admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:32:25.946489 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1443/2868] 10.222.0.23 () {60 vars in 1482 bytes} [Thu May 19 12:32:25 2022] PUT /identity/v3/projects/b0b15da1cdfb45c88bb7827e650d33c9/users/f901fc3ba9b24c39a252aa5a7c9d431c/roles/259eb400cdb4439db72d14456ae9cd08 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:32:25.952440 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317dfb55-3c92-4059-b718-727a2d6251f7 None None] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:32:25.952756 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317dfb55-3c92-4059-b718-727a2d6251f7 None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:32:25.953041 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317dfb55-3c92-4059-b718-727a2d6251f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:32:25.973854 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.auth.core [None req-317dfb55-3c92-4059-b718-727a2d6251f7 None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:32:25.977494 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-317dfb55-3c92-4059-b718-727a2d6251f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:32:26.052505 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1426/2869] 10.222.0.23 () {58 vars in 957 bytes} [Thu May 19 12:32:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 12:32:26.063956 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e056359-dbe5-4355-bcb8-866bf0ce7281 admin admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:32:26.065950 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-5e056359-dbe5-4355-bcb8-866bf0ce7281 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:32:26.132457 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e056359-dbe5-4355-bcb8-866bf0ce7281 admin admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:32:26.136930 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9721e13b-b0b1-449d-a02d-1644213e2f77 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a00b3fa7497456895e5e0bf105c4fc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd70d25bf69314820b8aaecb65576978a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:32:26.137732 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9721e13b-b0b1-449d-a02d-1644213e2f77 service neutron] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:32:26.138084 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9721e13b-b0b1-449d-a02d-1644213e2f77 service neutron] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:32:26.138407 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9721e13b-b0b1-449d-a02d-1644213e2f77 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:32:26.149178 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9721e13b-b0b1-449d-a02d-1644213e2f77 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:32:26.150764 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9721e13b-b0b1-449d-a02d-1644213e2f77 service neutron] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:32:26.162629 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1444/2870] 10.222.0.23 () {60 vars in 1365 bytes} [Thu May 19 12:32:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:32:28.084353 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e877c6-f78f-4a56-8fd9-6a0dbb2a33ee None None] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:32:28.085240 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e877c6-f78f-4a56-8fd9-6a0dbb2a33ee None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:32:28.085394 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e877c6-f78f-4a56-8fd9-6a0dbb2a33ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:32:28.134589 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.auth.core [None req-55e877c6-f78f-4a56-8fd9-6a0dbb2a33ee None None] MFA Rules not processed for user `f901fc3ba9b24c39a252aa5a7c9d431c`. Rule list: `[]` (Enabled: `True`). {{(pid=80539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:32:28.134589 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-55e877c6-f78f-4a56-8fd9-6a0dbb2a33ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:32:28.193371 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:32:28.238733 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1427/2871] 10.222.0.23 () {58 vars in 957 bytes} [Thu May 19 12:32:28 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:32:28.253428 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9721e13b-b0b1-449d-a02d-1644213e2f77 service neutron] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:32:28.256080 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-9721e13b-b0b1-449d-a02d-1644213e2f77 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:32:28.330411 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9721e13b-b0b1-449d-a02d-1644213e2f77 service neutron] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:32:28.334673 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14806aaf-523c-4fc1-9777-88b4b287612c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8800254c0e784e42a791fb9abc274d79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd70d25bf69314820b8aaecb65576978a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:32:28.335355 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14806aaf-523c-4fc1-9777-88b4b287612c service cinder] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:32:28.335646 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14806aaf-523c-4fc1-9777-88b4b287612c service cinder] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:32:28.335966 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14806aaf-523c-4fc1-9777-88b4b287612c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:32:28.347545 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14806aaf-523c-4fc1-9777-88b4b287612c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:32:28.349192 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14806aaf-523c-4fc1-9777-88b4b287612c service cinder] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:32:28.360619 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1445/2872] 10.222.0.23 () {60 vars in 1365 bytes} [Thu May 19 12:32:28 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 12:32:30.237375 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca756980-c189-4b84-9bc5-ab9fc42dbe1b None None] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:32:30.237974 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca756980-c189-4b84-9bc5-ab9fc42dbe1b None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:32:30.237974 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca756980-c189-4b84-9bc5-ab9fc42dbe1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:32:30.257597 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.auth.core [None req-ca756980-c189-4b84-9bc5-ab9fc42dbe1b None None] MFA Rules not processed for user `f901fc3ba9b24c39a252aa5a7c9d431c`. Rule list: `[]` (Enabled: `True`). {{(pid=80539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:32:30.260816 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-ca756980-c189-4b84-9bc5-ab9fc42dbe1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:32:30.267130 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1428/2873] 10.222.0.23 () {58 vars in 957 bytes} [Thu May 19 12:32:30 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:32:38.355042 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=59,keystone:INSERT=1 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:32:40.251724 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=14 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:33:09.749480 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14806aaf-523c-4fc1-9777-88b4b287612c service cinder] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:33:09.761799 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-14806aaf-523c-4fc1-9777-88b4b287612c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:33:09.763290 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14806aaf-523c-4fc1-9777-88b4b287612c service cinder] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:33:09.767315 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c5dfbe-77bc-4671-ac91-411d297f9645 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8800254c0e784e42a791fb9abc274d79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd70d25bf69314820b8aaecb65576978a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:33:09.768013 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c5dfbe-77bc-4671-ac91-411d297f9645 service cinder] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:33:09.768305 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c5dfbe-77bc-4671-ac91-411d297f9645 service cinder] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:33:09.768585 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c5dfbe-77bc-4671-ac91-411d297f9645 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:33:09.770548 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-86c5dfbe-77bc-4671-ac91-411d297f9645 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:33:09.858890 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c5dfbe-77bc-4671-ac91-411d297f9645 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:33:09.860394 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c5dfbe-77bc-4671-ac91-411d297f9645 service cinder] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:33:09.873475 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1446/2874] 10.222.0.23 () {60 vars in 1365 bytes} [Thu May 19 12:33:09 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:33:19.867516 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=34 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:33:40.321200 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca756980-c189-4b84-9bc5-ab9fc42dbe1b None None] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:33:40.332464 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca756980-c189-4b84-9bc5-ab9fc42dbe1b None None] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:33:40.336612 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ca9383-9cdc-42bf-a889-7339da76705e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8800254c0e784e42a791fb9abc274d79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd70d25bf69314820b8aaecb65576978a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:33:40.337982 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ca9383-9cdc-42bf-a889-7339da76705e service cinder] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:33:40.338816 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ca9383-9cdc-42bf-a889-7339da76705e service cinder] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:33:40.338816 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ca9383-9cdc-42bf-a889-7339da76705e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:33:40.340737 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-84ca9383-9cdc-42bf-a889-7339da76705e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:33:40.355722 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ca9383-9cdc-42bf-a889-7339da76705e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:33:40.357225 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ca9383-9cdc-42bf-a889-7339da76705e service cinder] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:33:40.370177 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1429/2875] 10.222.0.23 () {60 vars in 1365 bytes} [Thu May 19 12:33:40 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:33:49.536484 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c5dfbe-77bc-4671-ac91-411d297f9645 service cinder] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:33:49.547131 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c5dfbe-77bc-4671-ac91-411d297f9645 service cinder] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:33:49.551209 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f383374c-2b06-49e1-83d4-ab7cd5a8a1ce service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8800254c0e784e42a791fb9abc274d79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd70d25bf69314820b8aaecb65576978a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:33:49.551964 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f383374c-2b06-49e1-83d4-ab7cd5a8a1ce service cinder] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:33:49.552200 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f383374c-2b06-49e1-83d4-ab7cd5a8a1ce service cinder] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:33:49.552484 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f383374c-2b06-49e1-83d4-ab7cd5a8a1ce service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:33:49.554454 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-f383374c-2b06-49e1-83d4-ab7cd5a8a1ce service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:33:49.628314 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f383374c-2b06-49e1-83d4-ab7cd5a8a1ce service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:33:49.630261 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f383374c-2b06-49e1-83d4-ab7cd5a8a1ce service cinder] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:33:49.642392 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1447/2876] 10.222.0.23 () {60 vars in 1365 bytes} [Thu May 19 12:33:49 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 12:33:50.364232 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=6 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:33:59.636568 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=28 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:34:03.262165 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18dae8d3-5887-481a-8459-48b83de4a38c None None] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:34:03.262881 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18dae8d3-5887-481a-8459-48b83de4a38c None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:34:03.263019 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18dae8d3-5887-481a-8459-48b83de4a38c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:34:03.282909 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.auth.core [None req-18dae8d3-5887-481a-8459-48b83de4a38c None None] MFA Rules not processed for user `f901fc3ba9b24c39a252aa5a7c9d431c`. Rule list: `[]` (Enabled: `True`). {{(pid=80539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:34:03.284217 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-18dae8d3-5887-481a-8459-48b83de4a38c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:34:03.291482 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1430/2877] 10.222.0.23 () {58 vars in 957 bytes} [Thu May 19 12:34:03 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:34:03.573744 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f383374c-2b06-49e1-83d4-ab7cd5a8a1ce service cinder] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:34:03.588614 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f383374c-2b06-49e1-83d4-ab7cd5a8a1ce service cinder] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:34:03.592831 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562253a5-1c81-44c4-9d93-4c6a065a388a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a00b3fa7497456895e5e0bf105c4fc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd70d25bf69314820b8aaecb65576978a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:34:03.593640 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562253a5-1c81-44c4-9d93-4c6a065a388a service neutron] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:34:03.593935 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562253a5-1c81-44c4-9d93-4c6a065a388a service neutron] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:34:03.594212 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562253a5-1c81-44c4-9d93-4c6a065a388a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:34:03.596137 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-562253a5-1c81-44c4-9d93-4c6a065a388a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:34:03.607955 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562253a5-1c81-44c4-9d93-4c6a065a388a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:34:03.610990 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562253a5-1c81-44c4-9d93-4c6a065a388a service neutron] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:34:03.624798 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1448/2878] 10.222.0.23 () {60 vars in 1365 bytes} [Thu May 19 12:34:03 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 12:34:04.164003 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18dae8d3-5887-481a-8459-48b83de4a38c None None] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:34:04.166452 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-18dae8d3-5887-481a-8459-48b83de4a38c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:34:04.266861 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18dae8d3-5887-481a-8459-48b83de4a38c None None] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:34:04.272971 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d70871-eb69-417c-8c7b-8e1be6d24fda service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50f937eed4f3414fa6f19cab2a84eb5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd70d25bf69314820b8aaecb65576978a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:34:04.273782 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d70871-eb69-417c-8c7b-8e1be6d24fda service placement] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:34:04.274152 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d70871-eb69-417c-8c7b-8e1be6d24fda service placement] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:34:04.274428 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d70871-eb69-417c-8c7b-8e1be6d24fda service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:34:04.277926 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-86d70871-eb69-417c-8c7b-8e1be6d24fda service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:34:04.291296 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d70871-eb69-417c-8c7b-8e1be6d24fda service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:34:04.292984 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d70871-eb69-417c-8c7b-8e1be6d24fda service placement] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:34:04.304522 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1431/2879] 10.222.0.23 () {60 vars in 1365 bytes} [Thu May 19 12:34:04 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 12:34:08.570459 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562253a5-1c81-44c4-9d93-4c6a065a388a service neutron] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:34:08.572375 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-562253a5-1c81-44c4-9d93-4c6a065a388a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:34:08.586102 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562253a5-1c81-44c4-9d93-4c6a065a388a service neutron] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:34:08.591416 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be328ce-3dd5-4d18-86d7-289177bba82c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a00b3fa7497456895e5e0bf105c4fc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd70d25bf69314820b8aaecb65576978a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:34:08.592209 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be328ce-3dd5-4d18-86d7-289177bba82c service neutron] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:34:08.592582 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be328ce-3dd5-4d18-86d7-289177bba82c service neutron] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:34:08.592924 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be328ce-3dd5-4d18-86d7-289177bba82c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:34:08.594957 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-6be328ce-3dd5-4d18-86d7-289177bba82c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:34:08.613340 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be328ce-3dd5-4d18-86d7-289177bba82c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:34:08.615669 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be328ce-3dd5-4d18-86d7-289177bba82c service neutron] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:34:08.626896 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1449/2880] 10.222.0.23 () {60 vars in 1365 bytes} [Thu May 19 12:34:08 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:34:10.073746 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb899be-75b4-4d4f-9c0f-00771a01728e None None] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:34:10.074543 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb899be-75b4-4d4f-9c0f-00771a01728e None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:34:10.074543 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb899be-75b4-4d4f-9c0f-00771a01728e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:34:10.093584 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.auth.core [None req-bcb899be-75b4-4d4f-9c0f-00771a01728e None None] MFA Rules not processed for user `2d158b01a0a74de4bc0131b5c625efdf`. Rule list: `[]` (Enabled: `True`). {{(pid=80539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:34:10.094889 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-bcb899be-75b4-4d4f-9c0f-00771a01728e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:34:10.103591 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1432/2881] 10.222.0.23 () {58 vars in 957 bytes} [Thu May 19 12:34:10 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 12:34:18.622073 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=12 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:34:20.088413 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=38 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:34:35.614865 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be328ce-3dd5-4d18-86d7-289177bba82c service neutron] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:34:35.617065 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-6be328ce-3dd5-4d18-86d7-289177bba82c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:34:35.640018 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be328ce-3dd5-4d18-86d7-289177bba82c service neutron] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:34:35.650605 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d3e952-5f3c-4b50-8fb5-0721788a3fba service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34a98d75d84b4c67981377e0f26eca62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd70d25bf69314820b8aaecb65576978a', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:34:35.651996 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d3e952-5f3c-4b50-8fb5-0721788a3fba service nova] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:34:35.652492 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d3e952-5f3c-4b50-8fb5-0721788a3fba service nova] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:34:35.652987 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d3e952-5f3c-4b50-8fb5-0721788a3fba service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:34:35.656321 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-d3d3e952-5f3c-4b50-8fb5-0721788a3fba service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:34:35.679180 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d3e952-5f3c-4b50-8fb5-0721788a3fba service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:34:35.682324 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d3e952-5f3c-4b50-8fb5-0721788a3fba service nova] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:34:35.702335 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1450/2882] 10.222.0.23 () {60 vars in 1365 bytes} [Thu May 19 12:34:35 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:34:43.361425 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb899be-75b4-4d4f-9c0f-00771a01728e None None] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:34:43.373069 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb899be-75b4-4d4f-9c0f-00771a01728e None None] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:34:43.376736 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46794a4c-f12a-440b-a960-fd9ee48e1706 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a00b3fa7497456895e5e0bf105c4fc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd70d25bf69314820b8aaecb65576978a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:34:43.377386 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46794a4c-f12a-440b-a960-fd9ee48e1706 service neutron] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:34:43.377669 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46794a4c-f12a-440b-a960-fd9ee48e1706 service neutron] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:34:43.377945 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46794a4c-f12a-440b-a960-fd9ee48e1706 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:34:43.379841 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-46794a4c-f12a-440b-a960-fd9ee48e1706 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:34:43.456119 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46794a4c-f12a-440b-a960-fd9ee48e1706 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:34:43.457632 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46794a4c-f12a-440b-a960-fd9ee48e1706 service neutron] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:34:43.468992 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1433/2883] 10.222.0.23 () {60 vars in 1365 bytes} [Thu May 19 12:34:43 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:34:45.694169 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=6 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:34:46.265952 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d3e952-5f3c-4b50-8fb5-0721788a3fba service nova] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:34:46.267960 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-d3d3e952-5f3c-4b50-8fb5-0721788a3fba service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:34:46.283822 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d3e952-5f3c-4b50-8fb5-0721788a3fba service nova] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:34:46.291891 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe9b086-ea15-429b-8710-501cbe989458 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34a98d75d84b4c67981377e0f26eca62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd70d25bf69314820b8aaecb65576978a', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:34:46.292569 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe9b086-ea15-429b-8710-501cbe989458 service nova] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:34:46.292853 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe9b086-ea15-429b-8710-501cbe989458 service nova] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:34:46.293129 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe9b086-ea15-429b-8710-501cbe989458 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:34:46.295942 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-cbe9b086-ea15-429b-8710-501cbe989458 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:34:46.319373 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe9b086-ea15-429b-8710-501cbe989458 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:34:46.323122 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe9b086-ea15-429b-8710-501cbe989458 service nova] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:34:46.344683 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1451/2884] 10.222.0.23 () {60 vars in 1365 bytes} [Thu May 19 12:34:46 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 80 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:34:53.464133 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=30 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:34:56.335760 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=6 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:35:30.363171 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46794a4c-f12a-440b-a960-fd9ee48e1706 service neutron] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:35:30.386141 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46794a4c-f12a-440b-a960-fd9ee48e1706 service neutron] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:35:30.391516 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a9b4e5-1f44-4842-8712-5630ecdb043e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:35:30.393005 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a9b4e5-1f44-4842-8712-5630ecdb043e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:35:30.393472 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a9b4e5-1f44-4842-8712-5630ecdb043e admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:35:30.393910 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a9b4e5-1f44-4842-8712-5630ecdb043e admin admin] PATH_INFO: `/v3/users/f901fc3ba9b24c39a252aa5a7c9d431c` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:35:30.396203 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a9b4e5-1f44-4842-8712-5630ecdb043e admin admin] RBAC: Authorizing `identity:delete_user(user_id=f901fc3ba9b24c39a252aa5a7c9d431c)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:35:30.399963 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a9b4e5-1f44-4842-8712-5630ecdb043e admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:35:30.617245 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-96a9b4e5-1f44-4842-8712-5630ecdb043e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f901fc3ba9b24c39a252aa5a7c9d431c'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:35:30.633586 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-96a9b4e5-1f44-4842-8712-5630ecdb043e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f901fc3ba9b24c39a252aa5a7c9d431c'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:35:30.634617 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-96a9b4e5-1f44-4842-8712-5630ecdb043e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f901fc3ba9b24c39a252aa5a7c9d431c'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:35:30.706590 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-96a9b4e5-1f44-4842-8712-5630ecdb043e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f901fc3ba9b24c39a252aa5a7c9d431c'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:35:30.716807 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-96a9b4e5-1f44-4842-8712-5630ecdb043e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f901fc3ba9b24c39a252aa5a7c9d431c'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:35:30.724604 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1434/2885] 10.222.0.23 () {58 vars in 1223 bytes} [Thu May 19 12:35:30 2022] DELETE /identity/v3/users/f901fc3ba9b24c39a252aa5a7c9d431c => generated 0 bytes in 362 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 12:35:31.030681 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe9b086-ea15-429b-8710-501cbe989458 service nova] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:35:31.049457 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe9b086-ea15-429b-8710-501cbe989458 service nova] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:35:31.130820 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee1504d-072c-47a5-970a-bb1a8b435c81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:35:31.131776 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee1504d-072c-47a5-970a-bb1a8b435c81 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:35:31.132145 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee1504d-072c-47a5-970a-bb1a8b435c81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:35:31.132470 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee1504d-072c-47a5-970a-bb1a8b435c81 admin admin] PATH_INFO: `/v3/projects/b0b15da1cdfb45c88bb7827e650d33c9` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:35:31.134464 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee1504d-072c-47a5-970a-bb1a8b435c81 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b0b15da1cdfb45c88bb7827e650d33c9)` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:35:31.136557 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee1504d-072c-47a5-970a-bb1a8b435c81 admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:35:31.265116 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-1ee1504d-072c-47a5-970a-bb1a8b435c81 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b0b15da1cdfb45c88bb7827e650d33c9', 'request_id': 'req-1ee1504d-072c-47a5-970a-bb1a8b435c81'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:35:31.272371 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-1ee1504d-072c-47a5-970a-bb1a8b435c81 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b0b15da1cdfb45c88bb7827e650d33c9', 'request_id': 'req-1ee1504d-072c-47a5-970a-bb1a8b435c81'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:35:31.279821 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-1ee1504d-072c-47a5-970a-bb1a8b435c81 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b0b15da1cdfb45c88bb7827e650d33c9', 'request_id': 'req-1ee1504d-072c-47a5-970a-bb1a8b435c81'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:35:31.360021 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-1ee1504d-072c-47a5-970a-bb1a8b435c81 admin admin] The token cache is being invalidate because project b0b15da1cdfb45c88bb7827e650d33c9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80540) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 12:35:31.360323 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-1ee1504d-072c-47a5-970a-bb1a8b435c81 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:35:31.360994 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-1ee1504d-072c-47a5-970a-bb1a8b435c81 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:35:31.362714 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1452/2886] 10.222.0.23 () {58 vars in 1232 bytes} [Thu May 19 12:35:31 2022] DELETE /identity/v3/projects/b0b15da1cdfb45c88bb7827e650d33c9 => generated 0 bytes in 333 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 12:35:31.368960 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee38717d-c72b-448d-a4db-03278eb275c7 None None] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:35:31.370287 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee38717d-c72b-448d-a4db-03278eb275c7 None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:35:31.370468 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee38717d-c72b-448d-a4db-03278eb275c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:35:31.391871 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.auth.core [None req-ee38717d-c72b-448d-a4db-03278eb275c7 None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:35:31.393134 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-ee38717d-c72b-448d-a4db-03278eb275c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:35:31.466983 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1435/2887] 10.222.0.23 () {58 vars in 957 bytes} [Thu May 19 12:35:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:35:31.476114 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb79102-2489-4997-8558-96125084c0d5 None None] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:35:31.477093 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb79102-2489-4997-8558-96125084c0d5 None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:35:31.477516 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb79102-2489-4997-8558-96125084c0d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:35:31.499295 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.auth.core [None req-fcb79102-2489-4997-8558-96125084c0d5 None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:35:31.500586 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-fcb79102-2489-4997-8558-96125084c0d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:35:31.509209 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1453/2888] 10.222.0.23 () {58 vars in 957 bytes} [Thu May 19 12:35:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:35:31.513920 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee38717d-c72b-448d-a4db-03278eb275c7 None None] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:35:31.525477 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee38717d-c72b-448d-a4db-03278eb275c7 None None] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:35:31.530656 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333da9dc-d7f2-42aa-8e8a-e354db47852c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:35:31.531355 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333da9dc-d7f2-42aa-8e8a-e354db47852c admin admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:35:31.531645 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333da9dc-d7f2-42aa-8e8a-e354db47852c admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:35:31.531964 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333da9dc-d7f2-42aa-8e8a-e354db47852c admin admin] PATH_INFO: `/v3/domains` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:35:31.532757 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333da9dc-d7f2-42aa-8e8a-e354db47852c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80539) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:35:31.533075 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333da9dc-d7f2-42aa-8e8a-e354db47852c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:35:31.534753 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333da9dc-d7f2-42aa-8e8a-e354db47852c admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:35:31.549872 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1436/2889] 10.222.0.23 () {58 vars in 1152 bytes} [Thu May 19 12:35:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:35:31.562299 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb79102-2489-4997-8558-96125084c0d5 None None] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:35:31.579941 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb79102-2489-4997-8558-96125084c0d5 None None] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:35:31.588973 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb75a6e5-2c80-4b81-b480-fd9589513116 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:35:31.590177 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb75a6e5-2c80-4b81-b480-fd9589513116 admin admin] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:35:31.590674 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb75a6e5-2c80-4b81-b480-fd9589513116 admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:35:31.591160 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb75a6e5-2c80-4b81-b480-fd9589513116 admin admin] PATH_INFO: `/v3/projects` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:35:31.592496 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb75a6e5-2c80-4b81-b480-fd9589513116 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:35:31.596338 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb75a6e5-2c80-4b81-b480-fd9589513116 admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:35:31.721036 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1454/2890] 10.222.0.23 () {60 vars in 1152 bytes} [Thu May 19 12:35:31 2022] POST /identity/v3/projects => generated 391 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 12:35:31.725372 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333da9dc-d7f2-42aa-8e8a-e354db47852c admin admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:35:31.736693 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333da9dc-d7f2-42aa-8e8a-e354db47852c admin admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:35:31.827066 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31457b77-0605-451a-bf4b-e95bd3a5b92e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:35:31.830196 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31457b77-0605-451a-bf4b-e95bd3a5b92e admin admin] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:35:31.830196 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31457b77-0605-451a-bf4b-e95bd3a5b92e admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:35:31.830196 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31457b77-0605-451a-bf4b-e95bd3a5b92e admin admin] PATH_INFO: `/v3/users` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:35:31.830650 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31457b77-0605-451a-bf4b-e95bd3a5b92e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:35:31.834486 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31457b77-0605-451a-bf4b-e95bd3a5b92e admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:35:31.847650 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:35:31.951378 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1437/2891] 10.222.0.23 () {60 vars in 1143 bytes} [Thu May 19 12:35:31 2022] POST /identity/v3/users => generated 296 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 12:35:31.956384 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb75a6e5-2c80-4b81-b480-fd9589513116 admin admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:35:31.969030 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb75a6e5-2c80-4b81-b480-fd9589513116 admin admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:35:31.974672 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f637a8dc-7237-4cd8-86b0-1df53fecbd62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:35:31.975634 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f637a8dc-7237-4cd8-86b0-1df53fecbd62 admin admin] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:35:31.976090 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f637a8dc-7237-4cd8-86b0-1df53fecbd62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:35:31.976410 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f637a8dc-7237-4cd8-86b0-1df53fecbd62 admin admin] PATH_INFO: `/v3/roles` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:35:31.977725 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f637a8dc-7237-4cd8-86b0-1df53fecbd62 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:35:31.979676 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f637a8dc-7237-4cd8-86b0-1df53fecbd62 admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:35:31.998323 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1455/2892] 10.222.0.23 () {58 vars in 1121 bytes} [Thu May 19 12:35:31 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 12:35:32.003382 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31457b77-0605-451a-bf4b-e95bd3a5b92e admin admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:35:32.022250 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31457b77-0605-451a-bf4b-e95bd3a5b92e admin admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:35:32.031675 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28da832-94d1-4b75-b536-741f709cfa6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:35:32.036524 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28da832-94d1-4b75-b536-741f709cfa6c admin admin] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:35:32.036524 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28da832-94d1-4b75-b536-741f709cfa6c admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:35:32.036524 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28da832-94d1-4b75-b536-741f709cfa6c admin admin] PATH_INFO: `/v3/roles` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:35:32.036524 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28da832-94d1-4b75-b536-741f709cfa6c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:35:32.038409 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28da832-94d1-4b75-b536-741f709cfa6c admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:35:32.059830 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1438/2893] 10.222.0.23 () {58 vars in 1121 bytes} [Thu May 19 12:35:32 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 12:35:32.064877 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f637a8dc-7237-4cd8-86b0-1df53fecbd62 admin admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:35:32.076907 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f637a8dc-7237-4cd8-86b0-1df53fecbd62 admin admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:35:32.081905 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bced74-557e-44d6-a76a-0535dccbcb2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:35:32.082740 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bced74-557e-44d6-a76a-0535dccbcb2b admin admin] REQUEST_METHOD: `PUT` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:35:32.083025 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bced74-557e-44d6-a76a-0535dccbcb2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:35:32.083304 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bced74-557e-44d6-a76a-0535dccbcb2b admin admin] PATH_INFO: `/v3/projects/f05959730bfe477086d1d70e826b003b/users/72f9cc71cd56402b8ee2823eb4e9a8a3/roles/259eb400cdb4439db72d14456ae9cd08` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:35:32.101350 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1bced74-557e-44d6-a76a-0535dccbcb2b admin admin] RBAC: Authorizing `identity:create_grant(project_id=f05959730bfe477086d1d70e826b003b, user_id=72f9cc71cd56402b8ee2823eb4e9a8a3, role_id=259eb400cdb4439db72d14456ae9cd08)` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:35:32.104255 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1bced74-557e-44d6-a76a-0535dccbcb2b admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:35:32.171360 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1456/2894] 10.222.0.23 () {60 vars in 1482 bytes} [Thu May 19 12:35:32 2022] PUT /identity/v3/projects/f05959730bfe477086d1d70e826b003b/users/72f9cc71cd56402b8ee2823eb4e9a8a3/roles/259eb400cdb4439db72d14456ae9cd08 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 12:35:32.189402 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ce417e-adf4-416a-b6e5-c03da73e207d None None] REQUEST_METHOD: `POST` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:35:32.189855 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ce417e-adf4-416a-b6e5-c03da73e207d None None] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:35:32.190340 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ce417e-adf4-416a-b6e5-c03da73e207d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:35:32.240593 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.auth.core [None req-e6ce417e-adf4-416a-b6e5-c03da73e207d None None] MFA Rules not processed for user `72f9cc71cd56402b8ee2823eb4e9a8a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80539) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:35:32.243927 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-e6ce417e-adf4-416a-b6e5-c03da73e207d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:35:32.324078 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1439/2895] 10.222.0.23 () {58 vars in 957 bytes} [Thu May 19 12:35:32 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 12:35:32.335661 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bced74-557e-44d6-a76a-0535dccbcb2b admin admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:35:32.337509 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-e1bced74-557e-44d6-a76a-0535dccbcb2b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:35:32.360471 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:35:32.405645 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bced74-557e-44d6-a76a-0535dccbcb2b admin admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:35:32.409792 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d745ade0-07b8-4c14-baa9-a12f12439cad service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8800254c0e784e42a791fb9abc274d79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd70d25bf69314820b8aaecb65576978a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:35:32.410472 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d745ade0-07b8-4c14-baa9-a12f12439cad service cinder] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:35:32.410791 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d745ade0-07b8-4c14-baa9-a12f12439cad service cinder] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:35:32.411080 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d745ade0-07b8-4c14-baa9-a12f12439cad service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:35:32.422124 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d745ade0-07b8-4c14-baa9-a12f12439cad service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:35:32.423620 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d745ade0-07b8-4c14-baa9-a12f12439cad service cinder] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:35:32.434259 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1457/2896] 10.222.0.23 () {60 vars in 1365 bytes} [Thu May 19 12:35:32 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 12:35:42.319797 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:INSERT=3,keystone:SELECT=39 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:35:42.480249 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=18 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:36:00.590813 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ce417e-adf4-416a-b6e5-c03da73e207d None None] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:36:00.713969 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ce417e-adf4-416a-b6e5-c03da73e207d None None] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:36:00.720817 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc2b475-ca0f-4328-9e8a-7f8c48181eb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:36:00.722333 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc2b475-ca0f-4328-9e8a-7f8c48181eb1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:36:00.722839 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc2b475-ca0f-4328-9e8a-7f8c48181eb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:36:00.723323 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc2b475-ca0f-4328-9e8a-7f8c48181eb1 admin admin] PATH_INFO: `/v3/users/72f9cc71cd56402b8ee2823eb4e9a8a3` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:36:00.725477 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc2b475-ca0f-4328-9e8a-7f8c48181eb1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=72f9cc71cd56402b8ee2823eb4e9a8a3)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:36:00.729729 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc2b475-ca0f-4328-9e8a-7f8c48181eb1 admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:36:00.969636 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-ffc2b475-ca0f-4328-9e8a-7f8c48181eb1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '72f9cc71cd56402b8ee2823eb4e9a8a3'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:36:00.984064 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-ffc2b475-ca0f-4328-9e8a-7f8c48181eb1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '72f9cc71cd56402b8ee2823eb4e9a8a3'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:36:00.984861 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-ffc2b475-ca0f-4328-9e8a-7f8c48181eb1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '72f9cc71cd56402b8ee2823eb4e9a8a3'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:36:01.072679 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-ffc2b475-ca0f-4328-9e8a-7f8c48181eb1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '72f9cc71cd56402b8ee2823eb4e9a8a3'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:36:01.081517 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-ffc2b475-ca0f-4328-9e8a-7f8c48181eb1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '72f9cc71cd56402b8ee2823eb4e9a8a3'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:36:01.089591 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1440/2897] 10.222.0.23 () {58 vars in 1223 bytes} [Thu May 19 12:36:00 2022] DELETE /identity/v3/users/72f9cc71cd56402b8ee2823eb4e9a8a3 => generated 0 bytes in 500 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:36:01.094758 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b352ff8-b7cb-41c9-b73d-968e3aff59d0 None None] REQUEST_METHOD: `POST` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:36:01.095483 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b352ff8-b7cb-41c9-b73d-968e3aff59d0 None None] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:36:01.095483 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b352ff8-b7cb-41c9-b73d-968e3aff59d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:36:01.115657 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.auth.core [None req-7b352ff8-b7cb-41c9-b73d-968e3aff59d0 None None] MFA Rules not processed for user `11855e565a88475189d279de9469c271`. Rule list: `[]` (Enabled: `True`). {{(pid=80540) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:36:01.118307 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-7b352ff8-b7cb-41c9-b73d-968e3aff59d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:36:01.190209 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1458/2898] 10.222.0.23 () {58 vars in 957 bytes} [Thu May 19 12:36:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:36:01.201730 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc2b475-ca0f-4328-9e8a-7f8c48181eb1 admin admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:36:01.203240 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-ffc2b475-ca0f-4328-9e8a-7f8c48181eb1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:36:01.278198 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc2b475-ca0f-4328-9e8a-7f8c48181eb1 admin admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:36:01.282134 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9f53ed-84ea-41c0-a913-238f1bfcc5e0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a00b3fa7497456895e5e0bf105c4fc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd70d25bf69314820b8aaecb65576978a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:36:01.282818 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9f53ed-84ea-41c0-a913-238f1bfcc5e0 service neutron] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:36:01.283105 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9f53ed-84ea-41c0-a913-238f1bfcc5e0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:36:01.283384 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9f53ed-84ea-41c0-a913-238f1bfcc5e0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:36:01.295570 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9f53ed-84ea-41c0-a913-238f1bfcc5e0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:36:01.297172 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9f53ed-84ea-41c0-a913-238f1bfcc5e0 service neutron] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:36:01.310932 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1441/2899] 10.222.0.23 () {60 vars in 1365 bytes} [Thu May 19 12:36:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 12:36:01.799642 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b352ff8-b7cb-41c9-b73d-968e3aff59d0 None None] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:36:01.801389 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-7b352ff8-b7cb-41c9-b73d-968e3aff59d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:36:01.817427 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b352ff8-b7cb-41c9-b73d-968e3aff59d0 None None] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:36:01.826999 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23cbb7a3-80eb-424f-90da-9f723ce28c82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:36:01.829101 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cbb7a3-80eb-424f-90da-9f723ce28c82 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:36:01.829430 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cbb7a3-80eb-424f-90da-9f723ce28c82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:36:01.829693 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cbb7a3-80eb-424f-90da-9f723ce28c82 admin admin] PATH_INFO: `/v3/projects/f05959730bfe477086d1d70e826b003b` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:36:01.831222 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23cbb7a3-80eb-424f-90da-9f723ce28c82 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f05959730bfe477086d1d70e826b003b)` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:36:01.833429 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23cbb7a3-80eb-424f-90da-9f723ce28c82 admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:36:01.998989 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-23cbb7a3-80eb-424f-90da-9f723ce28c82 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f05959730bfe477086d1d70e826b003b', 'request_id': 'req-23cbb7a3-80eb-424f-90da-9f723ce28c82'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:36:02.006276 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-23cbb7a3-80eb-424f-90da-9f723ce28c82 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f05959730bfe477086d1d70e826b003b', 'request_id': 'req-23cbb7a3-80eb-424f-90da-9f723ce28c82'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:36:02.015584 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-23cbb7a3-80eb-424f-90da-9f723ce28c82 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f05959730bfe477086d1d70e826b003b', 'request_id': 'req-23cbb7a3-80eb-424f-90da-9f723ce28c82'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:36:02.141059 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-23cbb7a3-80eb-424f-90da-9f723ce28c82 admin admin] The token cache is being invalidate because project f05959730bfe477086d1d70e826b003b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80540) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 12:36:02.141747 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-23cbb7a3-80eb-424f-90da-9f723ce28c82 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:36:02.142117 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-23cbb7a3-80eb-424f-90da-9f723ce28c82 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:36:02.143939 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1459/2900] 10.222.0.23 () {58 vars in 1232 bytes} [Thu May 19 12:36:01 2022] DELETE /identity/v3/projects/f05959730bfe477086d1d70e826b003b => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:36:11.303956 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=79,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:36:12.137369 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=54,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:37:05.252047 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9f53ed-84ea-41c0-a913-238f1bfcc5e0 service neutron] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:37:05.254474 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-ff9f53ed-84ea-41c0-a913-238f1bfcc5e0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:37:05.389400 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9f53ed-84ea-41c0-a913-238f1bfcc5e0 service neutron] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:37:05.510424 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd45944-87a8-4ed3-95bd-9a364e80a460 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34a98d75d84b4c67981377e0f26eca62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd70d25bf69314820b8aaecb65576978a', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:37:05.511528 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd45944-87a8-4ed3-95bd-9a364e80a460 service nova] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:37:05.512056 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd45944-87a8-4ed3-95bd-9a364e80a460 service nova] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:37:05.512540 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd45944-87a8-4ed3-95bd-9a364e80a460 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:37:05.515735 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-3dd45944-87a8-4ed3-95bd-9a364e80a460 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:37:05.534935 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd45944-87a8-4ed3-95bd-9a364e80a460 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:37:05.537193 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd45944-87a8-4ed3-95bd-9a364e80a460 service nova] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:37:05.554873 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1442/2901] 10.222.0.23 () {60 vars in 1365 bytes} [Thu May 19 12:37:05 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:37:06.082043 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23cbb7a3-80eb-424f-90da-9f723ce28c82 admin admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:37:06.084098 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-23cbb7a3-80eb-424f-90da-9f723ce28c82 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:37:06.193515 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23cbb7a3-80eb-424f-90da-9f723ce28c82 admin admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:37:06.200775 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079c705d-0edc-4176-ade9-77b857c2b9c7 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50f937eed4f3414fa6f19cab2a84eb5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd70d25bf69314820b8aaecb65576978a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:37:06.201667 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079c705d-0edc-4176-ade9-77b857c2b9c7 service placement] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:37:06.202158 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079c705d-0edc-4176-ade9-77b857c2b9c7 service placement] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:37:06.202497 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079c705d-0edc-4176-ade9-77b857c2b9c7 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:37:06.204952 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-079c705d-0edc-4176-ade9-77b857c2b9c7 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:37:06.218921 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079c705d-0edc-4176-ade9-77b857c2b9c7 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:37:06.220848 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079c705d-0edc-4176-ade9-77b857c2b9c7 service placement] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:37:06.234153 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1460/2902] 10.222.0.23 () {60 vars in 1365 bytes} [Thu May 19 12:37:06 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:37:15.547577 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=62 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:37:16.228555 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=32 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:37:18.067166 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd45944-87a8-4ed3-95bd-9a364e80a460 service nova] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:37:18.068989 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-3dd45944-87a8-4ed3-95bd-9a364e80a460 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:37:18.140699 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd45944-87a8-4ed3-95bd-9a364e80a460 service nova] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:37:18.147010 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c64dae-9c24-49d0-8f3c-7010cf86541b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a00b3fa7497456895e5e0bf105c4fc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd70d25bf69314820b8aaecb65576978a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:37:18.148425 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c64dae-9c24-49d0-8f3c-7010cf86541b service neutron] REQUEST_METHOD: `GET` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:37:18.148931 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c64dae-9c24-49d0-8f3c-7010cf86541b service neutron] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:37:18.149490 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c64dae-9c24-49d0-8f3c-7010cf86541b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:37:18.152738 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-61c64dae-9c24-49d0-8f3c-7010cf86541b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:37:18.253044 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c64dae-9c24-49d0-8f3c-7010cf86541b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:37:18.255817 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c64dae-9c24-49d0-8f3c-7010cf86541b service neutron] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:37:18.277051 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1443/2903] 10.222.0.23 () {60 vars in 1365 bytes} [Thu May 19 12:37:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 210 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 12:37:27.331012 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079c705d-0edc-4176-ade9-77b857c2b9c7 service placement] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:37:27.332754 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-079c705d-0edc-4176-ade9-77b857c2b9c7 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:37:27.346949 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079c705d-0edc-4176-ade9-77b857c2b9c7 service placement] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:37:27.352658 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beed68af-61ed-4b63-8f88-a8e09db804bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:37:27.353445 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beed68af-61ed-4b63-8f88-a8e09db804bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:37:27.356203 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beed68af-61ed-4b63-8f88-a8e09db804bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:37:27.356203 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beed68af-61ed-4b63-8f88-a8e09db804bb admin admin] PATH_INFO: `/v3/users/2d158b01a0a74de4bc0131b5c625efdf` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:37:27.373002 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beed68af-61ed-4b63-8f88-a8e09db804bb admin admin] RBAC: Authorizing `identity:delete_user(user_id=2d158b01a0a74de4bc0131b5c625efdf)` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:37:27.375088 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beed68af-61ed-4b63-8f88-a8e09db804bb admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:37:27.561250 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-beed68af-61ed-4b63-8f88-a8e09db804bb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2d158b01a0a74de4bc0131b5c625efdf'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:37:27.575689 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-beed68af-61ed-4b63-8f88-a8e09db804bb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2d158b01a0a74de4bc0131b5c625efdf'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:37:27.576499 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-beed68af-61ed-4b63-8f88-a8e09db804bb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2d158b01a0a74de4bc0131b5c625efdf'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:37:27.630786 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-beed68af-61ed-4b63-8f88-a8e09db804bb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2d158b01a0a74de4bc0131b5c625efdf'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:37:27.639174 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-beed68af-61ed-4b63-8f88-a8e09db804bb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2d158b01a0a74de4bc0131b5c625efdf'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:37:27.646183 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1461/2904] 10.222.0.23 () {58 vars in 1223 bytes} [Thu May 19 12:37:27 2022] DELETE /identity/v3/users/2d158b01a0a74de4bc0131b5c625efdf => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:37:27.925504 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c64dae-9c24-49d0-8f3c-7010cf86541b service neutron] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:37:27.940998 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c64dae-9c24-49d0-8f3c-7010cf86541b service neutron] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:37:28.029433 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd510ad4-aec0-4bc6-b55d-031dfa57504c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:37:28.030382 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd510ad4-aec0-4bc6-b55d-031dfa57504c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:37:28.030735 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd510ad4-aec0-4bc6-b55d-031dfa57504c admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:37:28.031014 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd510ad4-aec0-4bc6-b55d-031dfa57504c admin admin] PATH_INFO: `/v3/projects/4b46044a7f1d471582800c9d8a79e5c5` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:37:28.044316 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd510ad4-aec0-4bc6-b55d-031dfa57504c admin admin] RBAC: Authorizing `identity:delete_project(project_id=4b46044a7f1d471582800c9d8a79e5c5)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:37:28.046395 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd510ad4-aec0-4bc6-b55d-031dfa57504c admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:37:28.167090 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:37:28.173640 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-dd510ad4-aec0-4bc6-b55d-031dfa57504c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4b46044a7f1d471582800c9d8a79e5c5', 'request_id': 'req-dd510ad4-aec0-4bc6-b55d-031dfa57504c'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:37:28.185417 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-dd510ad4-aec0-4bc6-b55d-031dfa57504c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4b46044a7f1d471582800c9d8a79e5c5', 'request_id': 'req-dd510ad4-aec0-4bc6-b55d-031dfa57504c'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:37:28.259579 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-dd510ad4-aec0-4bc6-b55d-031dfa57504c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4b46044a7f1d471582800c9d8a79e5c5', 'request_id': 'req-dd510ad4-aec0-4bc6-b55d-031dfa57504c'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:37:28.278276 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-dd510ad4-aec0-4bc6-b55d-031dfa57504c admin admin] The token cache is being invalidate because project 4b46044a7f1d471582800c9d8a79e5c5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80539) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 12:37:28.278771 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-dd510ad4-aec0-4bc6-b55d-031dfa57504c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:37:28.279845 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-dd510ad4-aec0-4bc6-b55d-031dfa57504c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:37:28.282444 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1444/2905] 10.222.0.23 () {58 vars in 1232 bytes} [Thu May 19 12:37:27 2022] DELETE /identity/v3/projects/4b46044a7f1d471582800c9d8a79e5c5 => generated 0 bytes in 358 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:37:28.288398 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beed68af-61ed-4b63-8f88-a8e09db804bb admin admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:37:28.290014 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-beed68af-61ed-4b63-8f88-a8e09db804bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:37:28.303954 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beed68af-61ed-4b63-8f88-a8e09db804bb admin admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:37:28.383696 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ea830e-8886-4887-8e63-12d2b59b2e8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:37:28.385392 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ea830e-8886-4887-8e63-12d2b59b2e8c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:37:28.386271 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ea830e-8886-4887-8e63-12d2b59b2e8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:37:28.386732 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ea830e-8886-4887-8e63-12d2b59b2e8c admin admin] PATH_INFO: `/v3/users/2d9f8b22c6e44f1784ac4ca285cfbcf5` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:37:28.412104 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ea830e-8886-4887-8e63-12d2b59b2e8c admin admin] RBAC: Authorizing `identity:delete_user(user_id=2d9f8b22c6e44f1784ac4ca285cfbcf5)` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:37:28.414289 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ea830e-8886-4887-8e63-12d2b59b2e8c admin admin] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:37:28.779063 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-87ea830e-8886-4887-8e63-12d2b59b2e8c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2d9f8b22c6e44f1784ac4ca285cfbcf5'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:37:28.784238 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=82,keystone:DELETE=17,keystone:INSERT=1 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:37:28.796398 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-87ea830e-8886-4887-8e63-12d2b59b2e8c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2d9f8b22c6e44f1784ac4ca285cfbcf5'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:37:28.797319 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-87ea830e-8886-4887-8e63-12d2b59b2e8c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2d9f8b22c6e44f1784ac4ca285cfbcf5'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:37:28.865156 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-87ea830e-8886-4887-8e63-12d2b59b2e8c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2d9f8b22c6e44f1784ac4ca285cfbcf5'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:37:28.874938 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.notifications [None req-87ea830e-8886-4887-8e63-12d2b59b2e8c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2d9f8b22c6e44f1784ac4ca285cfbcf5'} {{(pid=80540) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:37:28.881502 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1462/2906] 10.222.0.23 () {58 vars in 1223 bytes} [Thu May 19 12:37:28 2022] DELETE /identity/v3/users/2d9f8b22c6e44f1784ac4ca285cfbcf5 => generated 0 bytes in 594 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:37:29.167205 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd510ad4-aec0-4bc6-b55d-031dfa57504c admin admin] Authenticating user token {{(pid=80539) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:37:29.168847 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.fernet_utils [None req-dd510ad4-aec0-4bc6-b55d-031dfa57504c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80539) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:37:29.187437 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd510ad4-aec0-4bc6-b55d-031dfa57504c admin admin] Validating token access rules against request {{(pid=80539) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:37:29.264489 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196a412a-3302-4b0d-b857-b66131f32cee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11855e565a88475189d279de9469c271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3569780f2fe4b24b4109e2eee3c26fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80539) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:37:29.265377 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196a412a-3302-4b0d-b857-b66131f32cee admin admin] REQUEST_METHOD: `DELETE` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:37:29.265658 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196a412a-3302-4b0d-b857-b66131f32cee admin admin] SCRIPT_NAME: `/identity` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:37:29.265933 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196a412a-3302-4b0d-b857-b66131f32cee admin admin] PATH_INFO: `/v3/projects/fe5c73250c7c4b96920c3d5ec5678ff5` {{(pid=80539) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:37:29.278465 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196a412a-3302-4b0d-b857-b66131f32cee admin admin] RBAC: Authorizing `identity:delete_project(project_id=fe5c73250c7c4b96920c3d5ec5678ff5)` {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:37:29.280439 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196a412a-3302-4b0d-b857-b66131f32cee admin admin] RBAC: Authorization granted {{(pid=80539) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:37:29.392725 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-196a412a-3302-4b0d-b857-b66131f32cee admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fe5c73250c7c4b96920c3d5ec5678ff5', 'request_id': 'req-196a412a-3302-4b0d-b857-b66131f32cee'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:37:29.400764 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-196a412a-3302-4b0d-b857-b66131f32cee admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fe5c73250c7c4b96920c3d5ec5678ff5', 'request_id': 'req-196a412a-3302-4b0d-b857-b66131f32cee'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:37:29.481458 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-196a412a-3302-4b0d-b857-b66131f32cee admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fe5c73250c7c4b96920c3d5ec5678ff5', 'request_id': 'req-196a412a-3302-4b0d-b857-b66131f32cee'} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:37:29.493483 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-196a412a-3302-4b0d-b857-b66131f32cee admin admin] The token cache is being invalidate because project fe5c73250c7c4b96920c3d5ec5678ff5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80539) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 12:37:29.493948 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-196a412a-3302-4b0d-b857-b66131f32cee admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:37:29.494820 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG keystone.notifications [None req-196a412a-3302-4b0d-b857-b66131f32cee admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80539) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:37:29.497015 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: [pid: 80539|app: 0|req: 1445/2907] 10.222.0.23 () {58 vars in 1232 bytes} [Thu May 19 12:37:29 2022] DELETE /identity/v3/projects/fe5c73250c7c4b96920c3d5ec5678ff5 => generated 0 bytes in 330 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:37:38.962564 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:DELETE=3,keystone:SELECT=10,keystone:INSERT=1 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:37:39.490541 dv-cinder-804757-28-iscsi devstack@keystone.service[80539]: DEBUG dbcounter [-] [80539] Writing DB stats keystone:SELECT=63,keystone:DELETE=11,keystone:INSERT=2 {{(pid=80539) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:42:58.087453 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ea830e-8886-4887-8e63-12d2b59b2e8c admin admin] Authenticating user token {{(pid=80540) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:42:58.089221 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-87ea830e-8886-4887-8e63-12d2b59b2e8c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:42:58.301545 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ea830e-8886-4887-8e63-12d2b59b2e8c admin admin] Validating token access rules against request {{(pid=80540) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:42:58.418325 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5433293e-4081-4df9-9437-b0d639ea786b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50f937eed4f3414fa6f19cab2a84eb5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd70d25bf69314820b8aaecb65576978a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80540) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:42:58.419395 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5433293e-4081-4df9-9437-b0d639ea786b service placement] REQUEST_METHOD: `GET` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:42:58.419897 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5433293e-4081-4df9-9437-b0d639ea786b service placement] SCRIPT_NAME: `/identity` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:42:58.420399 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5433293e-4081-4df9-9437-b0d639ea786b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80540) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:42:58.424044 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.fernet_utils [None req-5433293e-4081-4df9-9437-b0d639ea786b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80540) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:42:58.447456 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5433293e-4081-4df9-9437-b0d639ea786b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:42:58.450800 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5433293e-4081-4df9-9437-b0d639ea786b service placement] RBAC: Authorization granted {{(pid=80540) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:42:58.472233 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: [pid: 80540|app: 0|req: 1463/2908] 10.222.0.23 () {60 vars in 1365 bytes} [Thu May 19 12:42:58 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:43:08.462904 dv-cinder-804757-28-iscsi devstack@keystone.service[80540]: DEBUG dbcounter [-] [80540] Writing DB stats keystone:SELECT=89 {{(pid=80540) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}